It's pretty straight forward to buy. I use Coinbase, but there are a bunch of different exchanges out there that will be more than happy to take your money. I chose Coinbase because it was based in the US (San Francisco) and have worked to comply with governmental agencies where required. The downside IS that they comply with governmental agencies.
Anyway, the other option is to purchase through something like GDAX.
It's a little more complicated, but the fees are much lower, if not free (if you're a market maker). For your first few transactions, I'd suggest just buying through an exchange as opposed to a trading platform.
With that said, let's assume you're going to use Coinbase. All you really need to do is create the account and make the purchase. I think you get limited to $500/week unless you give them a copy of your driver's license. Once I did that, I think they bumped me up to $25,000/week.
Once you purchase your Bitcoin / Ethereum / Litecoin it gets stored in your wallet. From there you can do anything you want with it. You can leave it there, transfer it to a different wallet, spend it, give it away, etc.
Keep in mind that you won't actually get anything. They aren't going to send you a piece of paper saying that you own x.xxxxxxx bitcoin. Similar to the stocks that I buy. They don't send out a certificate anymore, it's just held by my broker electronically. What you get is a public key, a private key and a wallet address.
With that said, we need to define a few things before we move forward.
The wallet address is like a bank account number. It's where you would send the Bitcoin if you wanted to buy my shoes or pay me for a pinball machine in bitcoin.
The private key is mathematically related to the wallet address, but impossible to reverse engineer. This private key is what allows you to sign a transaction and prove you own the coins and prevents the transaction from being altered. This is something you want to guard very closely and never give to anyone. I don't have mine written down anywhere, nor do I keep it on my computer or anything else that's online or connected (more on this later). Keep in mind that if you have $50,000 in bitcoin, all someone needs is the private key to send it all to themselves.
The public key is what you hand to people to during the transaction. It's derived from the private key, but once again, it's impossible to reverse engineer.
Now that we have that out of the way, we need to talk about wallet and storage types. There are two types of storage, a hot wallet and cold storage. A hot wallet is something that you use and have online access to. In most cases the wallet is stored on the exchange. This is where you keep bitcoins that you want to use to buy coffee, a computer from Microsoft, stuff from NewEgg, or buy your neighbor's lawnmower.
What that means is that you hold the public key and the exchange holds the private key. This is how it's dangerous. If the exchange gets hacked and someone gets your private key, game over. All your stuff is gone. It has happened to exchanges over the years, which is why I don't keep much in a hot wallet.
Cold storage is the exact opposite. It's not intended to be used for day-to-day transactions. It's an offline storage repository that has no connection to the Internet. It's quick to move Bitcoin from cold storage to a hot wallet, so don't get the wrong idea. More like a checking account at a bank (hot wallet) as opposed to a pile of cash in a the safe in your basement (cold storage).
The next option is cold storage. This is where you want to keep the majority of your investment. There are different types of cold storage, but what I use is a hardware wallet. There is also a paper wallet, where you can actually print out your bitcoins (weird huh?), but I never got into it so I can't comment much. I just know that I lose paper so it's probably not a good idea for me.
I use a Trezor. It's a hardware device that allows ME to hold the private key. Remember above, that when you keep it with an exchange, THEY hold the private key, not you. I like to be in complete control of my keys. You can also keep the private keys on your own computer in a wallet if you want, but that never sounded like a good idea to me.
When I buy / mine coins, I send them to my Trezor wallet address where the private key is stored on my device. I'm in complete control of my coins. Unless someone were to steal my Trezor and my PIN, or my recover seed, it's worthless. The recovery seed allows you to recreate your private keys if your hardware wallet were to be stolen / destroyed / lost. The PIN just allows you to move funds from your Trezor.
Whew, I think my fingers hurt.
Let me know if you have any questions or if there is anything I can clarify.