After lot of signed petitions. Trying lot of hacks which never worked. Even tried to implement my own imap server to gdata proxy (which was able to get the labels and mail headers), today, Google has made the dream of million of people true.
Since yesterday, GMail supports IMAP access. Yes, I tried it, and it works!.