The Raspberry Pi's most used model (B) was set to retail at $35. You have to add the keyboard/mouse/screen/power adapter (microUSB) and SD card. If you want Internet, you'll need a cable to a router or a wi-fi adapter.
The Raspberry Pi can run Linux (among other operating systems). Right now, you can't run Windows or Mac apps. But there's a lot of free software available for Linux, including office suites and games.
The desktop environment itself, and your applications and documents are all stored on a single SD card. This is the type of memory card that is common among digital cameras. If you and a friend wanted to share one Pi computer, you could just swap out memory cards when each of you wants to use it.
I will be writing some getting started guides and the like for the Pi in my spare time.