Well, take for instance, the UAE. They restrict everything on the Internet quite heavily. They wouldn't want someone buying a song that discusses a girl's desire to engage in premarital sex (Katy Perry's "Peacock"). So their firewalls block a huge amount of that, and Apple doesn't want to try circumventing that.
Yes, I mean in countries with lower technology levels, there is a huge reliance on older media, such as CDs. Now, I am not saying every former Soviet bloc country is a no-computer shithole, just saying that those nations are less online than others. So the digital craze hasn't hit them as much. Record labels are also very, very, very conservative, as we all know.
Apple is restricted by copyright law. They can't just offer their songs in countries without the label's permission. Then they would get sued, and they would lose.