Enough documentation is available about this on the internet, but for some reason I find that they are slightly fragmented and none of them give an end-to-end view. So here is an order in which you can go through these steps smoothly –
1) Follow the steps mentioned on this page – https://pages.github.com/ — This will activate your <github_username>.github.io page
2) For your custom domain to show/point to this URL, now follow the steps mentioned here – https://help.github.com/articles/adding-a-cname-file-to-your-repository/ — While configuring the DNS settings, pay special attention to whether your custom domain URL is going to be a TLD(Top Level Domain) URL or a sub domain URL. You need to only one of the two steps mentioned in that article, i.e. you setup an A record, or a CNAME record. Unless you have some special use-case, do not setup both(I cant imagine when that can make sense). A friend of mine ended up setting up both records and got a 400 Nginx Bad Request when attempting to access his website URL on the browser!
In this context, this Stack Overflow reply – http://stackoverflow.com/a/22374542/183902 – helped in troubleshooting the CNAME and A record mix up for my friend.
3) This should get you up and running with a Hello World website. Now install jekyll from Ruby Gems and structure and style your pages! If you are looking for some quick setup, or are better at editing CSS than writing from scratch(like me :D), then you can take a look at these two themes that are good starting points – https://github.com/poole/hyde and https://github.com/poole/lanyon . Just fork and clone these repos and use the files for your site, with or without customizations.
There your Custom Domain website, powered by Github Pages is ready!
Thanks to my friend’s attempt to setup his site, I ended up reviving http://vatsala.co.in from the dead. Only a landing page there right now, need to bring myself to set up the full site.