The C64 starts up straight into BASIC from ROM. Unlike some other contemporary computers, it doesn't attempt to boot from any external devices (except the cartridge port). There isn't really a DOS in the usual sense. Apart from simple support for loading and saving programs, and a very basic channel I/O facility, everything else is handled by the firmware in the disk drive, which has its own 6502 and operating system.
For example, there's no command for getting a directory listing. You type `LOAD "$",8` (8 being the disk drive), and the drive pretends there's a BASIC program called `$` that happens to contain a directory listing you can then look at with `LIST`. (https://en.wikipedia.org/wiki/Commodore_DOS#/media/File:Comm...)
By default, LOAD loads tokenized BASIC programs, but if you add an extra `,1` to the command, the file can contain arbitrary data starting at any location in memory. You could use this to load a machine language program and then run it with `SYS <location>`. Clever programmers figured out they could skip this step by having their file overwrite a vector that gets called after the load completes and jump right into their code, resulting in every Commodore kid having being able to type `LOAD"*",8,1` on autopilot.
I got distracted by other trivia (I grew up with this computer and it was hugely influential and I will adore it forever) from getting to the actual point: The C64 uses a variant of the 6502, the 6510. It has a special register for swapping out any combination of the three ROMs (BASIC, KERNAL (sic), and the character ROM) plus I/O registers that overlay portions of the 64 address space. If your code doesn't use those, you can access the RAM they are hiding by turning them off.
On my ATARI there was no DOS too. When you start the 65XE you can hold (iirc) START to start loading an application from the cassette recorder, but it was recommended to hold both (again iirc) START and OPTION to bypass BASIC, because BASIC interpreter being held in the memory, somehow interfered with bigger games (I think this was due to memory, but I'd like to learn from someone who know). I myself got into this trouble sometimes. Also you could have a CARTDRIGE with DOS-like Turbo management which allowed to scan cassette for given filename with binary application, but no one used this because it would take crazy long. I never had chance to use floppy disk, but I think it was behaving in a similar way (you had to have a floppy with DOS and hold START when powering the computer to load it), but at that time the FDD drives for atari were horryfyingly expensive (they had the same CPU 6502, and even there were some demoes which used this CPU as a coprocessor), so I stayed with a cassete reader with TURBO.
Of course games were also sold on CARTDRIGEs and this was the fastest way to play, but it wasn't popular in my country.
The original Atari 400/800 included BASIC on a ROM cartridge.
To use BASIC, you plugged the BASIC cartridge into the system and powered up.
To boot something else (games...., from either cassette or disk) you first removed the cartridge, then powered up.
With the XE series, BASIC was built in to the console, so the "magic keys" were needed to tell the hardware to virtually "unplug" the BASIC ROM before it tried booting from any connected devices.
Yes, Option would disable Basic on boot. The first Ataris (400 and 800) came with a basic module that you had to put in and then start the computer to use Basic - or likewise Assembler. The module would then use certain parts of the precious 64 KB Ram - actually, much less because the OS in ROM would write itself into RAM on startup and take about 20KB away. So a program or game had about 40 KB space to use. Basic would take some more away. Which wasn‘t a problem on the 800, you would either plug the module in and use Basic but you wouldn‘t if you wanted to load a game. But with the XL, you needed a way to disable the automatic Basic load at boot time, or many games could not use all of the memory they needed. Hence, the Option-option at startup.
At least I remember it this way, but I only had an XL, not the older ones, and now I remember that the 800 had only 48KB of RAM, so it was probably more complicated than that!
> much less because the OS in ROM would write itself into RAM on startup and take about 20KB away.
RAM shadowing of the ROM did not exist in the Atari's (at least not in the original 400/800 models). The ROM's simply were physically connected to actually "be" the top 16KB of the 6502's 64k max address space. The CPU executed the ROM code by directly reading the code from the ROM chips.
Which is also the reason the original 400/800 models were limited to 48k max RAM. 16k of the address space was already used by the ROMs.
