Question about New School Web Authentication and Images

Time was, you authenticated via htaccess, and the server handled the details for you. If you gave the right password, you could access the materials in a directory, and if you didn't, you couldn't.

Now, we're authenticating with OAuth with Google or the like, and the smart stuff you write knows whether you're authenticated or not, but that kinda happens at the client level, and your server doesn't know whether it's authenticated or not.

xkcd

If you're talking to the database to get content, there has to be some code there that says "OK, there's authentication, so we know Alice's content from Bob's, but if there's something saved to file, like an image — You could put your images in a database table, and I've done it, but in practice it's kinda stupid — that means that image is hardcoded, so anyone could browse around your authentication.

So, is the solution to have the images in a non-web-shared directory and have code determine whether it's OK to send the image or not? I've done that, too, but it seems like you stop the server from doing what it's good at. As efficient as you can make a program that takes in a user identifier, determines if it's acceptable, reads in a file and sends either a good image or a "that's not okay" image, that's always going to be slower than letting your server just send that image.

So, do I own that slowness? Is there another way that I just don't know yet? I'd put this on Stack Overflow if I even knew how to pose the question. Any thoughts?