ATX is the typical form factor used for motherboards by today's modern desktop computers. Basically, if your motherboard is ATX, you need an ATX case (and power supply). A case has space for a power supply (often times one is included, although it may not be enough watts to power your system. The front has space for the optical drives. You can also customize your case, adding things like special lighting if it pleases you.
The most popular CPUs are made by Intel or AMD. If you want to use more than 3GB of RAM in your system, it's best to go straight to a 64-bit CPU (32-bit CPUs cannot address large amounts of RAM). Of course, you'll need an OS in the 64-bit version as well to take advantage of the extra RAM. That said, you can pick single-core, dual-core, quad-core, or other CPU configurations. Depending on your needs, a single-core or dual-core CPU may be sufficient.
If you're planning to save money by skipping a graphics card, you can look for a CPU with "integrated graphics."
The motherboard you get needs to match the socket your CPU will fit. So, if you pick an Intel CPU, you'll need a motherboard with a chipset compatible with that specific CPU. Likewise, if you choose AMD, you'll need a motherboard compatible with that specific CPU. Motherboards can often include on-board USB2/USB3/3.1 ports, audio, network (LAN/Ethernet), Bluetooth and Wi-Fi. Each specific board offers a certain amount of these features.
The amount of RAM you need really depends on your specific needs. Generally, 1GB total memory should only be used if you're using the system for very light tasks. Desktops can be build in a good price range with 4GB or more RAM.
If you're planning to use more than 3GB, remember to get a 64-bit CPU and a 64-bit operating system.
For some, purchasing a CPU that also includes "integrated graphics" may be sufficient for their needs. PC builders often add a graphics card later if integrated graphics is the only thing in their budget when they start their build. That said, graphics cards can easily be added and upgraded on an specific available port on the motherboard.
There are a very, very wide range of video cards, typically ranging from 1GB of memory on up. This is one part, along with the CPU, that I highly suggest you spend quite a bit of time researching before making your purchase.
If you use the integrated graphics with a CPU with that feature, you'll probably be borrowing from system memory. If the on-board chip is configured for 1GB of video RAM, you'd subtract the 1GB from the total memory, leaving what's left as available to your operating system.Example, your system has 4GB of RAM, but uses integrated graphics at 1GB, you would only have 3GB of RAM available to the operating system. Buying a seperate "discrete" graphics card prevents this borrowing.