Make Coveo For Sitecore work on Authenticated Sitecore URL

You have your Sitecore Instance URL protected.  Why? Many organizations do this to protect their under development site to be indexed or even worse seen by search engines and obviously you dont want any one who knows the URL to actually access your not so ready site or worse add a reference link or something.
So, yes, it is very common to have this kind of set up on lower environments while development is in progress.  It could be to share this URL with selected Clients for UAT or content load in some cases.
You have Coveo for Sitecore installed on this Sitecore instance that you protected using IIS authentication.
The Problem – “Coveo For Sitecore will not work – Diagnostics will fail 401, Index rebuilding will not work(401 again!) and obviously your renderings will not load as Indexes are not built yet”
Solution – While scrambling for a good solution and still leave the Sitecore Instance URL protected, we actually did find more than one solution through various unrelated sources and one from Coveo team as well . 🙂

Hopefully, this will help some one who is still scrambling and unable to find a solution.  Select which way to implement and go with based on your usage of Coveo Indexes.

1. If your project uses plain old Coveo renderings and not use Coveo Indexes from Sitecore content search – You can choose to add another IIS site on CD server and point that to the same site, use this URL(internally open and no auth) to manage your coveo indexes. 
2. If your project accesses Coveo indexes from code and content search – #1 wont work and will cause 500(Coveo rest end point to pull index will error due to 401 again). In this case, you have two options 
• Disable IIS basic auth and use IP whitelisting for security instead as our infrastructure thinks this is a possible
•  Keep IIS Authentication enabled on all folders of the Sitecore instance except WebsiteCoveorest and Website/Sitecore Modules/Web/Coveo