See the magic – Coveo Recommendations

If you have not gotten a chance to review my other posts that I wrote up based on our experience implementing Coveo recommendations from the scratch, I would strongly advise going through them first.

Post 1 – Was all about visual clues

Post 2 – The real deal

In this post, now that we have gotten all the set up done, I will focus on how to actually get some recommendations to show up to ensure we apply styles, test the filters out and basically yeah see a working version of recommendations. It was not actually that straight forward, but, thanks to couple hands and also huge thank you to @vbernard to help give some awesome tips that helped us force recommendations when everything else failed.

First, when we have the rendering up on the presentation on all our product detail templates where we wanted to show recommendations from Coveo, we started by visiting lot of pages that made sense to the user flow. I had couple of hands, no kidding here, so, basically I read few posts to understand how Coveo recommendations work internally, though it is a black box, it is fairly easy to understand the basic logic behind how it works. So, we attacked it in such a way that it is bent to give us some results.

This is what we did and it worked on our lower environment and started sending recommendations back, you can also choose to use your QA to fire off some tests and automated requests to do the same. We did a bit of both actually – manual go to set of links that emulated various levels of our product pages and an automated hit to several urls as well.

In a nut shell, Coveo recommendations looks for patterns based on history of the end user and suggests what might interest the user next based on the history and matches that it can find from the recommendations model. It is also important to ensure recommendations model has enough data by adjusting the training settings on the ML model. We tried couple different settings there and rebuilding the model after we visited bunch of pages, circling around the products we care about. Finally! We see the below on our recommendations component

We got lucky to be able to test this on lower environments due to multiple hands and effort by the team to emulate what we could, so, we can test it thoroughly with filters on hand. Oh! In regards to filters, we actually needed lot of context based filters, so, we approached this in two fold process –

  1. Adding a processor via config to coveoProcessParsedRestResponse, this essentially would add in some additional meta data per result that we need for queries and filters. There is some real good example and documentation loaded here . We followed similar path, but, added our own business logic to load context data needed on our filters
  2. Next up, we added a JS query filter as documented here and used a bunch of expression builders based on all the data we loaded either on processor or via the recommendations view such as current item ID or some miscellaneous information we needed per business requirements on what to show on recommendations and what not to.

Now, I am a true believer of back up plans, if our hardwork to emulate and get the recommendations engine to send us a few recommendations had failed, then, we would have applied styles and do other tests via a hack provided to us by @vbernard from Coveo. Here are the steps to make that work:

Step 1: Add a new pipeline

Step 2: Push some featured results to this pipeline

Ignore the cool stones and take a peek at the featured results

Step 3 : Now go back to ML models and add a recommendations model

Thats it! You should now see the recommendations forced and you would probably see the featured results on your recommendations. Do note though that you should start really small/basic on your view if you are trying to do this forced result approach. In our case, it was kind of hard to use this approach and test what we needed to as we really lean on context for most part. But, probably it might work on basic use cases.

Coveo new GUID generation issue

Thursday last week, one of my team member ran in to an issue which was preventing XHR request to fire off to Coveo and get the results back. He struggled to overcome the issue almost whole day long and at that time as his lead it became my responsibility to help solve the issue. I was actually quite doubtful about it and unsure if I could help solve or not, but, as a lead, rule is you can not pass on fear to your team, you have to no matter you feel pass confidence that – ‘Yes, we will get over this’. Challenge Accepted!

Here is a sneak peek of the issue. So, in older versions of Coveo and in more fond memory on Legacy framework implementation, each interface element had a unique ID that you would need to pass from presentation details via properties on your rendering. If you do not do much about it, a unique identifier will be generated when you create a new content item that has the presentation of Coveo embedded. With hive, it is actually not so different, pretty similar, only the properties are on data sources. The issue on hand is that, on the interface view HTML, the @Model.ID was actually coming back as ‘$GenerateNewUniqueID’ instead of DOM Element ID noted on presentation. You can see this below on the screenshot.

Error on JS for generating unique ID.

So, as with any issue, it becomes very important to replicate the issue on local especially when you can not get your brain juices flowing on call on tough problems like this. See the steps I did to crack this thing

  1. Why would Coveo not replace this in first place? It should automatically happen. My team mate stumbled upon few Q&A and checked the pipeline related that does this behind the scenes to ensure that is in place and hooked properly.
  2. We checked the presentation again very carefully and datasources to ensure DOM ID is present and is unique
  3. Ensure this is copied over properly to Final Layout on Sitecore as well and indeed there is a correct data source being passed in presentation.
  4. Then we ran an ugly experiment to actually replace standard value as it seemed like Coveo for some reason was taking that. No luck, still same issue.
  5. Re-installed Coveo For Sitecore package relating to our Sitecore version and published everything. Still Nothing!
  6. This was working just fine on our UAT server, so, just to try our luck ported over all packages from UAT related to bare bone Coveo templates, layouts, etc., Nope! Did not work.
  7. Ran out of all options!

One little detail from my team mate sparked an experience from the past. He said when he installed a package that has just one item while he was experimenting did something and though the XHR request was still having lot of issues, he could now see the ID replaced now. That reminded me that in the past when we ran in to weird issues where presentation was not updated even if the presentation did look right on Coveo end, an IIS Reset had solved all problems. Went ahead and tried that and knocked out browser cache just in case and boom, everything started working.

I should have probably done IIS reset first before trying anything, but, with more knowledge you gather you would want to explore that rather than blind belief that a restart or reset would solve things. 🙂

This whole debugging experience got me thinking if I was not a Software Engineer, I would have made one awesome FBI Agent for some reason. It almost felt like I was connecting from past and paying attention of little detail on hand to solve the issue. Quite a few memories of great crime investigation series flashed in my brain. lol

Localization of your search page

speak the language

Couple weeks ago I was psyched to start a series on our first hand experience dealing with Coveo for Sitecore on a multi-tenant/multi-site Sitecore instance.  Check out my first two posts below:

Link Manager Gotcha
Get Your Links right

Now, last, but, not least when you have such huge instance of Sitecore, you are bent to have things you would want to localize due to multiple languages you are supporting.  I usually see the solution in two folds.

  1. Coveo OOTB component localization
  2. Custom string localization – Like Facets or any strings on your search listing card that are computed field for instance.


The first one is quite simple and easy to do as Coveo carries responsibility OOTB to load most close culture based on context language.  If you go to viewsource on your search page, you will see that Coveo has already done some magic for you.  It loads the culture *.js file based on your context language.    This happens right the view based on below line of code on Coveo Search View. 

<script type="text/javascript" src="/Coveo/js/cultures/@(Model.CultureName).js"></script>

That should take care of your result summary on left top corner above your results.   For your sorting components which you may have added using your DateSortView or RelevancySortView, to localize the sorting header such as “Relevancy” or “Date” in context language, Coveo looks for special data attribute on HTML called ‘data-caption’ and if your HTML element for your facet has this attribute loaded with your title and if that is a existing key on locales dictionary object of your interested culture/language then it is respected and corresponding localized string is shown on your page. 

Now, for the second fold, if you would like to localize your Facet values, Headers, etc., There are few things to consider/answer – Do you show just one language content on your search page per site?  If you are – Life is simple, just ensure you have localized content loaded on your Facet field.  If your facet is based on computed field, ensure you are loading language specific value based on currently indexed item language.   For your facet header to be localized, you have couple of options here as well, you can either read it from your localized dictionary on Sitecore if you have one, if not, load up your JS localize dictionary with new values as noted in the link below.

https://docs.coveo.com/en/421/javascript-search-framework/changing-the-language-of-your-search-interface

Now, the problem actually happens when you have more than one language content show up on your search page, in that case, you will have an issue with mixed values which obviously is not UX friendly.    You can see a good solution and huddle between me and my favorite Sitecore MVP and Coveo champ below(@jflh ). 🙂

https://answers.coveo.com/questions/16338/localize-facet-values-on-multi-site-solution-using.html?childToView=16358#comment-16358

This is the last piece of this series and I am sure there will be more to come, this one is near and dear to my heart and the amount of learning and information I gathered through this Implementation is priceless.  Although, Coveo and Sitecore never fail to amaze me, there is more room to learn and grow with these products every single day. 😉

Happy Coveo for sitecore implementation to every one!

Some helpful links:

https://developers.coveo.com/display/public/JsSearch/Localization

https://docs.coveo.com/en/421/javascript-search-framework/changing-the-language-of-your-search-interface

 

Coveo and Sitecore Link Provider

links

My last post was a kick off a series of things to watch out for and pay attention to while implementing Coveo For Sitecore on a Multi-tenant/Multi-Site Sitecore instance.   This is a runner-up for my last post and part of same series.  Let me start this by saying, URL on search result is single most important entry point for your user to see detailed content around what they think is relevant.  To ensure this is rendered right every single time,  It is very critical that you know your Sitecore Link provider or Custom Link provider if any in and out on your Coveo For Sitecore installation.

There are few things that you will need to know in order to fine tune anything or in order to debug issues if any when you see things you do not wish to see in relation to Url’s either on Indexes or on your search page.  If you are lucky and do not have much of customization going in your Sitecore link provider, you should be fine doing nothing as Coveo fires off link provider at it’s two most important phases OOTB.

  1. Index Sync Process
  2. Displaying paged results on Coveo driven search page

Now, the most important thing to remember is that while Coveo does #1 noted above it calls Sitecore Link Provider in “shell” context while doing it’s best to load up UrlOptions object that is passed in as a parameter to Sitecore GetItemURL method of Link Provider with appropriate site details it could resolve the currently indexed item with.   Coveo has their algorithm that runs as part of ResolveItemSiteProcessor located in coveoResolveItemSite pipeline.  Below is the snippet of that processor located on Coveo.SearchProvider.Config

<coveoResolveItemSite>
<processor type="Coveo.SearchProvider.Processors.ResolveItemSiteProcessor, Coveo.SearchProviderBase" />
</coveoResolveItemSite>

If this processor is not working wonders for you, go ahead and remove this in your favorite custom coveo configuration file and add your own.   For curious brains,  I recommend changing your level of logging to DEBUG to understand what happens step by step while Coveo tries syncing your Index.    That was very insightful in understanding sequence of calls and pipelines that fire while Coveo is indexing or doing sync process.  But, do remember change that back to INFO after your appetite is satisfied. 😉

Now, for the last clue,  on #2 stated above when your Coveo Search Page renders your search results for a query, it calls Sitecore link provider again to get the click uri for each search result that goes with each search result.  Now, pay attention here, very important to know that the context site this call runs in is “coveorest”.  If your link provider is super awesome and written very well thought out, you will have no problems regardless on the context site the call is running in.  But, if your link provider is using context.site instead of using site info from UrlOptions or doing some thing really out of this world, you have to know these things to ensure you hit the gold.

Hopefully these tidbits will help you debug any issues while Coveo constructs the URL’s at indexing or run time.  If you have lot of shared content that does not reside under a specific site node, then, it is very hard to get the URL right at indexing time which is why it is even more important to focus on #2 to ensure the end user experience is seamless and correct leading them to correct site, language and context.

There, I gave away a hint, next up, I will be covering language related insights like fall back and localization in my next post.  That is all for now, tune in soon for more.  🙂

 

 

 

 

 

Implement Coveo on Multi Tenant/Multi Language Architecture

I am very excited to start this series as we are almost close to pulling off Coveo Cloud Implementation on a multi site/multi language Sitecore Instance using legacy Coveo For Sitecore Javascript Framework to support web forms Sitecore Implementation.  I am gathering my thoughts around all the challenges we faced while getting Coveo to work seamlessly with complex Sitecore Instance.  Some were very straight forward and some were anything but that. 🙂

I am going to list down each important problem and note options to tackle each.  Do note that though most of the solutions I would attempt to provide are generic, it highly depends on your Sitecore implementation and customization done to Sitecore in general.

One thing that really helped me solve some of these is understanding when and where Coveo connects with Sitecore.   I would recommend to deep dive on this to any one who is attempting to implement Coveo For Sitecore on their multi tenant and multi lingual Sitecore implementation.

I will start by noting the first problem we faced right after Coveo for sitecore installation on our system.  On Diagnostics page, we had the error such as noted in question and answer below.

https://answers.coveo.com/questions/15435/coveo-search-rest-endpoint-invalid-uri-the-format.html?childToView=15530#comment-15530

One of the first question to answer when implementing Coveo For Sitecore is – Does your Sitecore implementation have a custom link manager as that drives what you would need to watch out for or customize if needed, a quick way to check would be

-> Go to your /sitecore/admin/showconfig.aspx and examine what you see in the node below.  if you see default provider as something other than “sitecore”, then, you can ensure that there has been some customization done on this.  To be honest, any medium to high sized Multi tenant instance that has been active on Sitecore for long time would have some customization around it.  So, it is even more important to know how to ensure Coveo works with your custom link provider settings.

link provider section on config

For instance, in our case languageembedding setting on Linkprovider was set to “always” which means that any website/site served by Sitecore, when a link is generated it would append language to it.  That should surely ring some bells, in  Coveo For Sitecore case upon installation of package, you would see three new sites in the <sites> node when you take a peek at showconfig which are named as coveo_website, coveoanalytics, coveorest.  We definitely do not want to add language embedding on these URL’s here to ensure we do not throw off internal business logic on Coveo end.

To gracefully solve this, I would recommend having an ability to use different Link Providers per site.  There is a very decent solution compiled by my favorite Sitecore MVP Jammy Kam below.

Site Specific Link Provider for Multisite Implementation in Sitecore

By doing this we can assure that Coveo specific sites do not have language embedding in it that would cause issue noted on Q&A above.

Of course, if your sitecore implementation was done more on old school style where the answer was – There is a pipeline for that. 🙂

In those cases, check to see where you can ensure to exclude Coveo sites from having language embedding on them.

One open question I do have is would this be any different with Sitecore 9.0+, a short answer might be a NO.  If any one has any input in regards to this, leave a comment.

That is it about our first ever hiccup, more to come soon.  My hope is that this series will help any one who is driven to use Coveo for Sitecore components and power of framework on a complex Sitecore instance, but, are worried that it might be quite a feat or they might be safe using total custom system.  You can re-visit my blog on this topic and I am sure you will get confidence in sticking to framework instead.  🙂

Tune in for more on this series!

 

 

Experiment with Ease

background experiment

When you are stuck in a problem, do not get scared to experiment!  You might be successful at solving the problem or not, but, every time you let go of your fears and experiment, you end up learning some thing new every single time.

On one of our heavy custom implementation with Coveo due to super complex architecture which uses Web forms, I exactly had to to that.  Funny story,  I am so used to MVC that literally when I dream of a solution I do so, you guessed it right in MVC. 🙂

So, we had a situation where we need to use Coveo Search Box component on shared content which could span across multiple sites and hence the URL should not be based on item and should rather be based on context.  While looking for a solution I bumped in to the below Q&A, but, the solution was for MVC.

https://answers.coveo.com/questions/9939/how-to-configure-coveo-search-box-view-rendering-o.html

I wanted to see how I can implement this on web forms and searched to find a parallel solve for the same with no luck.   So, the only option left was to experiment to see if it will actually work.

To prove my theory, I opened up couple Coveo dll’s to see how I can use Inheritance to swap the Model.  Turns out, it is possible to swap the Model out with few extra steps than MVC.  Do note, that in MVC the model swapping could be done on the View Rendering item to use your custom coveo search box model instead of OOTB coveo model.

So, below are the extra steps for web forms to achieve what is noted for MVC on the Q&A.  We tried these below and it works perfectly alright. 🙂 Rest of the steps should be similar to what is noted on the Q&A noted above, so, I will not repeat the awesome steps that Jean-François L’Heureux provided.

  1. In your duplicated Custom Coveo Search Box sublayout, let us call it CustomCoveoSearchBox.ascx change the inherits to your own custom class
    Inherits=”Coveo.UI.CoveoSearchbox”
    Would become
    Inherits=”ProjectName.Search.Extensions.CustomCoveoSearchbox”
    Use your best judgement or project guidelines to decide a namespace for your custom coveo search box class. 🙂
  2. Next step is to pass in your CustomSearchboxModel (a new custom class that would inherit SearchboxModel) in the OnLoad override of your CustomCoveoSearchBox  class.   Your CustomSearchboxModel would provide your new implementation for GetSearchPageUrl() that would have logic to determine the URL based on site context and other decision factors.

A quick screenshot from Dotpeek of Coveo.UI that helped me with the decisions.

CoveoSearchModel

Hope this helps some one battling the situation on web forms based Sitecore instance.

 

 

Coveo Cloud Learnings – Part 3

Follow my trail if you have not  read coveo cloud part 1 and 2.  Get a read on them here –

Coveo Cloud Learning – Part one 

Coveo Cloud Learning – Part two 

This is the last blog of this series, but, of course there is no end to learning.  I will continue to post new blogs as and when I encounter an interesting thing to share with you all as I proceed to take on new challenges and projects.

On this blog, I will cover how we used underscore, organized our views for better maintainability and all of this with Helix in play.

First though, Coveo does do an awesome job of documenting how to split result templates on conditions which comes in handy in various json situations.  Learn more here:
https://docs.coveo.com/en/413/javascript-search-framework/result-templates

After some pondering, we did go with data-condition route on our result templates.  We also have to remember to have a default result template at the last in the pile.  For better maintainability, we chose to add in partial views in MVC.  We do not have dynamic views as we only have a set of types.  If your implementation calls for content author control over these, you might choose to provide that additional layer of flexibility.  For us, it is an overhead in various ways, so, we avoided that and loaded the partial views on our own custom coveo search view.

Some thing like below:

@Html.Partial(“~/Views/Search/ResultTemplates/ProductResultTemplate.cshtml”, Model)
@Html.Partial(“~/Views/Search/ResultTemplates/StandardResultTemplate.cshtml”, Model)
@Html.Partial(“~/Views/Search/ResultTemplates/GeneralResultTemplate.cshtml”, Model)

Also, we definitely needed some same key attributes on all of this views and we do not want the Controller code to actually hit every single time, it would be in-efficient.  So, we chose to have a custom coveo model that inherits from the Search Model available in Coveo.UI.Mvc.Models with the additional attributes we need.   It was quite easy to make this happen, also, these instructions noted here might come in handy if you are trying something similar.   Awesome post by Vincent. 🙂 Basically, chose bits and pieces I needed to pull this off.

https://source.coveo.com/2014/12/10/creating-custom-model-coveo-for-sitecore/

Now, we have manageable pieces, so, we can work parallel on different result templates and also great for the future when it is time for enhancements.   In real life situations, it becomes very important while working with underscore templates to ensure a specific json piece you are looking for is not undefined and has a value in it, we had quite a few requirements to not even show a matching label on the view if a specific json attribute is missing.  We accomplished that with a simple underscore check as below on those applicable.  Below is a snippet for your reference.

{{ if (!_.isUndefined(raw.yearoforigin) && !_.isEmpty(raw.yearoforigin) ) { }}
<li class=”coveo-product-details__item”>
<span class=”coveo-product-details__category”>@Translate.Text(“Year of Origin”)</span>
<span>{{=raw.yearoforigin}}</span>
<a href=”#” class=”more”>(+)</a>
</li>
{{ } }}

Also, note that we continue to follow Sitecore best practices on the side to load all labels from Dictionary instead of hard coding on the view.  Another thing to note, is for the data conditions you need on the result templates, if it is a static value such as a GUID or string, always follow Helix best practice of having these defined in Templates.cs file of your project.  In our case, we maintained this in Feature Search project.  So, the data condition and templates.cs would look like –

<script id=’faqResultTemplate’ class=’result-template’ type=’text/underscore’ data-condition=’raw.contenttype==”@Templates.ContentTypes.FAQType”‘>

public struct ContentTypes
{
public static readonly string FAQType = “FAQs”;
}

I know, I wish this is not a string and is a GUID, but, unfortunately due to content structure we had to work with has these distributed out in content areas rather than defined templates, so, had to go this route.  But, where you can always rely on GUIDS as that is more stable implementation.

Also, we had lot of external fields which need to be used as Facets in our case.  There is documentation on how to make an external field Facetable, but, one thing to remember is actually a combination that you will need to do to make this work.  You have to do what is noted here and also ensure you add the field to external fields section as well.  As always,  it is always recommended to patch using a special config file on your Helix project rather than updating either custom coveo config  or raw coveo config file.

Say, if I had a field on my Cloud Index called contenttype, I would then add the following on this patch config file to ensure it is facetable and hash is excluded considering this is an external index.

<configuration xmlns:patch=”http://www.sitecore.net/xmlconfig/”>
<sitecore>
<coveo>
<defaultIndexConfiguration>
<fieldMap>
<fieldNames hint=”raw:AddFieldByFieldName”>fieldType fieldName=”contenttype” isExternal=”true” settingType=”Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework” />
</fieldNames>
</fieldMap>
<fieldMap>
<externalFields hint=”raw:AddExternalField”>
<field fieldName=”contenttype” />
</fieldMap>
</defaultIndexConfiguration>
</coveo>
</sitecore>
</configuration>

Also see this interesting post below that we used to migrate all our external fields on to production sandbox when we are ready.  We had lot of custom fields added to sandbox due to what we were trying to accomplish and wanted to eliminate manual work to load these up and replicate to avoid error and mundane task.  Thanks to coveo team for letting us know a better way does exist.  Take a peek of that here

Another interesting thing I bumped in to while working on this project was how some functionality is abstracted out from us on Cloud as compared to on-premise.  On Cloud, especially when working with trail sandboxes, if you queue way too many rebuilds, it might take a while, so, do not panic and give it some time.  All should be fine unless there is something else going on, if your logs are clean and no obvious errors on cloud console logs, you should be fine and it is just a delay on the cloud queue as trial sandboxes might not be given immediate preference.  If in doubt, talk to your Coveo point of contact and they will calm you down. 🙂

Also, when you are in active indexing state, say items are being added to your external index on cloud.  Sometimes, same queries and parameters passed from your Coveo Search Page/Components could yield different results due to elastic nature of Coveo.  Obviously, this should not happen when indexing is done.  But, I was super curious to know the reason behind this as I was doing some smoke test on my end.

That is all for now you all, have fun and enjoy the week ahead. Happy Monday!

Leave me a comment if you are curious to know more.

 

 

 

 

 

 

Coveo Cloud Learnings – Part 2

search

Last week, I kick started a planned series in effort to pour over my side of story using Coveo Cloud platform first hand.  You can read more on the initial piece here

I am hoping this series will help any one starting a Coveo cloud implementation in making appropriate choices along the way based on project and requirements on hand.  I would also post catches/gotchas that I experienced along this journey as well.

Now,  let us move on to more.  After the initial steps, there are more choices to make such as below.  I wish I could say – go with your  gut, lol.  Always go by a thoughtful and knowledgeable call on each of these.  When in question, use rock solid support from Coveo and Sitecore team.   You can ping on slack channel or load a Q&A here .  More than likely, if your decision is a basic one, there could be one already out there. 🙂

Ensure to squeeze out as much as you can from the OOTB product, Coveo Cloud offers Machine learning and AI that could help users find relevant results.  So, embrace it and try to make it work to your architecture and timing.   In our case, we thought query suggestions are very important.  But, what to do when there is not enough data available on Coveo Analytics that would get back ML query suggestions.  Ensure you always have a back up plan!  We turned on result suggestions with Omnibox component as a back up.  Although, be very careful and diligent to drop this off as these queries are accounted towards the quota you signed up for with Coveo.

A parallel note on ML/Query Suggestions and reminder to ensure to change the tuning parameters on your test sandbox to test your ML query suggestions.  If you leave them to default, it might or not bring up query suggestions.   If you do not know where to find them, go to your Coveo Administration console either via sitecore or direct link if you have one on you, then, click on Query Pipelines under Search section, pick a model you want to tune the settings on and you will see the screen below

trainingset

I also was curious to see and test the ML query suggestions, so, I followed the instructions noted here and could quickly create a new report of my own way to track if there are any ML query suggestions available on my model.

https://onlinehelp.coveo.com/en/cloud/reviewing_suggested_queries_by_coveo_machine_learning.htm

Our Architecture was subtly different as it uses older/legacy Sitecore version to push all the data on to Cloud index using Push API.  We had to ensure we are following all the good practices as much as we can to ensure Coveo can then perform and do it’s thing on relevancy.  Here is a good Q&A link and responses I got when I was eyeing and planning to ensure we are covered with this Architectural decisions we have in play.
https://answers.coveo.com/questions/14602/best-practices-when-using-push-api.html

Another little POC that I had to do on my side to ensure our Architecture decisions will not have any unknown risk on main requirements. One such requirement was to have Coveo Stand Alone search box to be available and working on older version of Sitecore site and it would then redirect to the new version of Sitecore search page that was built off of Coveo components and JS framework.  Initially, I had hiccups due to CORS issue as the domain was not matching, but, later all was cool when we had set up load balancer to swap between old and new sitecore versions.   Below is Q&A reference for this step as well, if you are curious
https://answers.coveo.com/questions/14985/can-we-have-a-coveo-component-html-on-a-site-while.html

Next up, I will add my notes around how we built the components and used underscore and any other caveats we encountered during the same.

That is all for now folks, if you need more information on this subtly different architecture we have in play, feel free to drop in a comment.   Have a great weekend! You all deserve it. 🙂

Coveo Cloud Learnings – Part 1

Coveo_Cloud

I am super excited to share my experiences with you as I rolled up my sleeve to play with super cool search tool out there, Coveo Cloud!

Before I got this chance to work intimately with this platform, I did hear a lot and read a lot.  Was wondering and anticipating, when will I get to actually do hands on and I really wished to do that before I head out to Coveo Impact at SanFrancisco this year. Between, crazy excited to hear all the awesome new things and also geared up to participate in their Hackathon event as well.  Know more about their event here below

http://impact.coveo.com/

Now, I plan to write up my experiences with Coveo cloud first hand, the project is near completion on my end and I think this is the perfect time to reflect on all the set of challenges we had while pulling off a simple yet complex due to Architecture decisions involved on the project.

First things first, when you are about to start implementation on Coveo for Sitecore platform, decide on your version and framework you plan to use.  This goes long way to ensure you picked the correct version that supports all your other related third party components and ensure it is indeed a right step towards desired search experience.

On this project, we chose December 2017 version matching the Sitecore version we are using on this instance.  My first comment on getting Coveo for Sitecore up and running on my local and other environments is “Piece of Cake”.  Compared to challenges, hiccups I and my fellow team members had with on premise version of Coveo, this was a total opposite.  It took me less than an hour as opposed two full days(give or take) with on-premise version.  It was an awesome feeling to see the green quite quickly this time.

While you are at this step in your project, also, decide which version of JS framework would you like to use.  I would vote for Hive as it is new, more robust and modular, but, it highly depends on project of course.   You can see compatibility and more decision factors here on their documentation.

https://developers.coveo.com/display/public/SitecoreV4/What%27s+New+In+Coveo+for+Sitecore+4

While I was doing the steps or while trying to understand the new platform, I tried comparing and contrasting to the knowledge I had working with their on-premise version.  It did help me a lot actually trying to figure – “How is this different from before?”

Some helpful notes on Installation and Set up with Cloud.   Do note that your sitecore indexes, both, master and web would be now on Cloud and so are your logs, content browser and other cool things you were used to pulling up from port 8081, it should now all be located on Coveo Cloud Console. Also, understand that some features are now abstracted out, like you can not turn off and on the live monitoring on indexes for instance.  One important thing to realize is all your index rebuilds are queued in some fashion which you may not have control over, so, talk to your Coveo consultant or drop in a support ticket if you encounter some delays more than usual on your rebuilds.

On our end, we always ensure to use cloud sandbox(trial) for all our lower environments.  Note that there could be some limit on number of indexes on Sandbox, so, check with your Coveo rep when in question.

One subtle catch I would like to mention is – after installation of Coveo for Sitecore and hooking up proper Coveo Sandbox configuration and a full site publish, I did see the web index on the cloud from Sitecore perspective, but, note that master did need a manual rebuild by going in to your Indexing Manager on Sitecore Control Panel.

If you got time on you, explore the Coveo Cloud Administration Console, it has different and clearly named sections that should give you fair idea on which is what, but, it is definitely worth going over this as you drill deep in to Cloud Implementation.

Also, note that with Cloud you can build sample search pages on the fly, it is a cool feature to test some of the fields you have on your index or could be a custom build external index that you would like to target, but, if your implementation calls for some thing more complex, I would recommend to house your search pages within your sitecore instance by customizing the components and adding them to your presentation plus creating the templates.

So, you installed Coveo For Sitecore cloud version, got all green in diagnostics, sitecore indexes in check and could see your items and indexed data on cloud console content browser.  Did I hear that right? That is it, we can now jump right ahead and do some cool stuff that is made possible by the platform.

My next blog will focus on more challenges, gotchas and learning we had during this wave of time.   Also, will expound on choices we made at each step as well.

References/Helpful Resources for Coveo For Sitecore Cloud 

Installation guide:
https://developers.coveo.com/display/public/SitecoreV4/Installing+Coveo+for+Sitecore

Architecture and Beyond
https://developers.coveo.com/display/public/SitecoreV4/Understanding+the+Architecture+of+Coveo+for+Sitecore

Framework Choice

https://developers.coveo.com/display/public/SitecoreV4/What%27s+New+In+Coveo+for+Sitecore+4

Coveo Impact

http://impact.coveo.com/

Versions

https://developers.coveo.com/display/public/SitecoreV4/Downloads

A Battle – Coveo Search Framework | Coveo Search API

I am going to give it away to you all in who is the winner. 🙂

In reality, I think there is no battle when there is a clear winner and your vision to pursue should always be to use Coveo for Sitecore platform.  When the world turns up side down, you hit the wall like 100 times and when Coveo team also gives up only then you would pick the other option to build ground up.

Folks who love to code to solve problems or passionate to take full control in to their own hands in terms of business logic implementation have to remember and think about why did the executives spent money and time investing in Coveo For Sitecore.  Now, with the cloud version, their biggest selling point is of no suspense, heavy contexual marketing and machine learning are sure a huge cherry on top to their easy to integrate search experience.  Between, if you have installed Coveo for Sitecore on-premise and then also had experience with Coveo For Sitecore cloud, you will have to agree with me.  It took minute slice of earlier to set up cloud platform on a Sitecore instance.

Recently, after we did a thorough comparison between these two options for one of our clients, Coveo team released their best practices, I included the link here.  If you have not already, please check this out, it is very useful.  I definitely learned and shared a thing or two from here with my teams.  Now, this document just solidified what we pitched and I could not be more happier. I mean getting a backing from Technical Architects from Coveo on what we thought was correct approach.  Nice feeling you all!

Coveo Best Practices Document:  https://developers.coveo.com/display/public/SitecoreV4/Coveo+for+Sitecore+Project+Guide

Hint:  Look at Page Number 52 – Section Developing Against the Coveo Search API directly

I would like to share pros/cons of these approaches for any one who is trying to convince their client to not lean towards re-building something they don’t have to. 🙂

To keep this to the point, I will list what you will miss if you go with using Coveo Search API instead of Coveo for Sitecore framework.

Disclaimer:  Got to Hurt…Ouch!!!

  1. You will forgo ability to be able to add in Coveo Custom Components, they are becoming more and more powerful especially if you have Enterprise and above license where you hardly would need to write any JS, just use properties as much as you can to pass in those Filtering and other settings.
  2. Incremental refresh of Indexes – Depending on how huge your solution is, this will hurt you more. 😉 Coveo when integrated with your sitecore deeply you will notice that it taps on to most important pipelines to ensure it keeps your master and web indexes in sync using some clever strategies.  That means it will not rebuild the whole indexes until you request it to do so, saving lot of time and being efficient.  Web sources and other type of indexes you might create might not have this feature.
  3. Machine Learning/Coveo Analytics and any thing beyond.  Now, let me re-iterate the obvious, the world is moving towards AI for better or for worse and this might be one the biggest reasons why Coveo was a big win and hence the decision to go forward with the product.  Coveo Analytics are collected through one of their component that when added to search interface will push all the goodness ML would need to then use awesome Coveo ML features such as query suggestions and ART for instance.  You will miss out on this if you do not install the framework.  Now, I do see some API’s with which you can get usage analytics and either do a read or write to Coveo Analytics using the same, but, why go through this process when you can simply add a component that has been built and tested by Coveo.  If you are curious –

https://usageanalytics.coveo.com/docs/write/

https://usageanalytics.coveo.com/docs/read/

There could be more?  you think you know some? Leave a comment.

Now, remember all this losses while making a decision, but, also do understand that in some really complicated architectures where there seem to be no way that we can seamlessly integrate and use full power of Coveo platform then Coveo Search API in combination of a lot of custom code might be a way.  But, before you go that route I urge you talk to every one you think might know a different, better or easier way to fully juice and extract the goodness of the platform.  At the end of the project, we definitely do not want regret on our plate on this type of decisions.

Now go out there and make some tough decisions. 🙂

Custom Sorting on Coveo Facet

I got to use a new feature of Coveo today because of new requirements. How fun, though the request that came through felt not needed and a corner case in terms of user experience. Pushing through limit is what we have to do if you have got super picky clients. 🙂
The outcome is always precious as you get to learn something new. Here is the snapshot of what happened and how we were able to get it working per request and requirement.

Goal – Regardless of selections made on Coveo Facet because of url appending, the sort order of values available on facet stay per the field on which sort is based on.

Problem – The inherent default behavior of Coveo for better UX experience. To be honest, I actually like what they have in terms of grouping the values selected to the top and rest are sorted based on the sort field set up. But, we have to some how overwrite this default behavior.

Firstly, our front end contact explored why this happens and how coveo groups them the way it does inherently. The reason for special attention that these selected values get is because of the below. Thanks to our front end resource to find this out. Though we did not pursue anything in modifying this, it is always good to know the reasons behind how it is currently done.

https://developers.coveo.com/display/public/SearchREST/Group+By+Results

So, the reason is that when there is an appending in URL for a selection of a value or set of values in the facet, Coveo inherently tags these as Desired and hence the special treatment for those to be on the top.

For example, if your URL is like below may be because the user clicked on back button post user selected few values and moved on his journey, the user could land on pages like this or if marketing team linked off the user to a page with pre-selected values on a facet.  There could be more scenarios where your user would bump in to this situation, but to name a few.

Example URL – https://www.domain.com/press-releases-events-and-webcasts-landing#f:articlePostYear=[2017,2018]

The selected would be grouped on top like below due to the above explained reason.

Again, I like what Coveo does inherently and I suspect that this needs to overridden in simple cases, but, if you absolutely need to do this? Which is where we stood, I logged a Q and A below and luckily got a direction.  I only wish there were more graceful options, but, we took what we got as well, we had to overwrite this and there was no other option unfortunately.

If you are curious, follow the discussion here and I will explain more in detail how I actually implemented what was recommended by Coveo team.

https://answers.coveo.com/questions/12071/facet-values-mentionedgrabbed-in-allowed-values-be.html?childToView=12086#answer-12086

Steps I did to Implement this solution:

  1. Created a copy of default Coveo MVC view for facet and named it differently and placed under correct project specific location.
  2. Add the below line to the view created just above end of div tag in place for the facet
    data-custom-sort=”2024,2023,2022,2021,2020,2019,2018,2017,2016,2015,2014″>
  3. Note – With addition of this, we should be good on sorting with lower limit of 2014 to upper limit of 2014, based on your project data you could decide on good numbers for these limits. Only deal is post 2024, if this project has same implementation, we will need a code deploy to ensure sort works as expected.
  4. Use this new view item on presentation for the facet and select all appropriate values that a coveo facet would need on the templates affected or will use such type of facet
  5. That is it!! Post this change regardless if values are selected via url or not, the ordering will be based on data-custom-sort.

Deeper dive in to Coveo Advanced Expressions

In our implementation we stumbled upon looking at what Coveo calls ‘aq’ bunch of times.  It is simply because of complexity involved on our implementation where we had to bend Coveo a little to achieve what the business needs are.

First up, let us first understand a little what is aq, dq and cq per Coveo.  They have beautiful documentation that iterates what these are.
https://developers.coveo.com/display/public/SearchREST/Query+Parameters

So, every time a query is fired by Coveo rendering because of coveo components you have on the page.   All these query parameters based on what is applicable is generated by Coveo, you can see this on Network tab in your browser when you take a look at Request headers and parameters.  If what coveo generates satisfies your business goals, you are good to call it a day.  But, what if it is not enough, you have to then modify and tweak to achieve what is needed.

When it comes to modifying the aq generated gracefully it becomes a challenge specifically if you are having issues with what Coveo does inherently while computing ‘aq’ for a rendering/request
One such case happened recently, where we needed a disjunction to be applied at a specific piece of generated ‘aq’.  We obviously could not get away with disjunctive expressions, I talked about this in my previous blog if you are curious.

https://deepthikatta.blogspot.com/2016/11/coveo-facet-slider-across-two-fields.html

The reason why this approach would not work in this case is because the way disjunctive expression works and is interpreted by Coveo in final query is aq OR dq which means in result set you will find all items that match your filters appended by Coveo in aq OR that match your expression that you added on dq.

But, what if you want this disjunction to applied only on a particular facet or custom advanced expression.  How can we do this? is it possible?

Like always, everything is possible. 🙂  In our case, it is little more complex because we also did not want what Coveo typically injects as a piece of aq on a Facet Slider for example because we would fall in to same problem which we began to solve if we don’t do this.   So, to achieve this, we did two steps.

1.  OR where you need – In event called buildingQuery, access the args and append the advanced expression that you need including a specific OR condition you care about.  Some thing like
Coveo.$(‘#@Model.Id’)
.on(“buildingQuery”, function(e, args) {
let mySlider = Coveo.$(‘#mySlider’)
if (mySlider != undefined) {
let optionsOfComponent = mySlider.coveo().getSliderBoundaryForQuery()
if (optionsOfComponent != undefined && optionsOfComponent[0] != undefined && optionsOfComponent[1] != undefined) {
let fieldOneQuery= ‘@(Model.ToCoveoFieldName(“field one”)) >=’ + optionsOfComponent[0] + ‘ AND @(Model.ToCoveoFieldName(“field one”)) <=’ + optionsOfComponent[1] let fieldtwoQuery= ‘@(Model.ToCoveoFieldName(“field two”)) >=’ + optionsOfComponent[0] + ‘ AND @(Model.ToCoveoFieldName(“field two”)) <=’ + optionsOfComponent[1]
args.queryBuilder.advancedExpression.add(‘((‘ + fieldOneQuery+ ‘) OR (‘ + fieldtwoQuery+ ‘))’) }}})

2. Remove the advanced expression added by Coveo –  Reverse engineered the string that Coveo would append on the facet slider per configuration and removed it using helper provided under Expression Builder.
https://developers.coveo.com/display/public/JsSearch/ExpressionBuilder

Some thing like –
.on(“doneBuildingQuery”, function(e, args)  {
//reverse engineer the value you would like to remove and ensure you have proper checks on coveo      components or facets before you access the values
let  expressionToRemove  = ‘@(Model.ToCoveoFieldName(“your slider         field”))==’+optionsOfComponent[0]+’..’+optionsOfComponent[1]
args.queryBuilder.advancedExpression.remove(expressionToRemove)})

That is it, we got what we needed when took a peek at Network tab and request parameters and the results matched to business goal.   One less thing to worry about.  We are launching this thing soon, totally psyched!

When you need more than filtered results

Recently, in one of our major Coveo renderings, there was a splurge of new requirements.  Well, no huge surprise there I think.  When powerful people meet in the same room, new ideas do come by, not such a good news to us developers who have scratched their brains and pulled up hours to wrap up the initial implementation.  Although, I believe it is a good venue when time is on your side to learn something new. 🙂

This is exactly what happened to us, the requirement sounds pretty simple, but took me few hours to find an optimum solution which solves the problem, but, also would not involve too many changes to existing logic in play.
Goal –   Get all the results that match the filters(facets) selected by end user with out the default pagination applied (Coveo has default pagination and only pulls the values entered in Number of results value given on the rendering properties).   After few hours of going over Coveo documentation, I found something though completely not the same as what I was looking for, it kind of sparked the similar wave of implementation.

I almost always start a Q&A either to confirm my thoughts or to see if there are any other ways to accomplish the same.   In this case, what i was going for just felt right and Coveo team member confirmed the same.  You can follow the trail here.

https://answers.coveo.com/questions/10368/do-we-have-allnot-limited-to-resultsperpage-result.html

So, basically to achieve what I was looking for, you need to fire a parallel query with exact same filters based on user interaction and simply change the results to an upper bound.  It would have been nice to actually set it to Totalresults, I have not tried that, but, could work when you get the value from the correct variable.

Here is a snippet –

Coveo.$(‘#<%= Model.Id %>’).on(Coveo.QueryEvents.querySuccess,function(e, args)
{
var query = args.query;
query.numberOfResults = 2;
//Do any other modifications on the query
Coveo.SearchEndpoint.endpoints[“default”].search(query).done(function(data) {
//do any other logic on the data up here
}
});
});

That is it, we have everything we need plus a handler over all results.  I am pretty sure it would come up often hopefully it helps any one else.

Happy problem solving folks. 🙂

Coveo Sorting Part 3

Hey There!

This time soon enough, I really wanted to complete my thoughts on Coveo sorting.  We so far learnt two main things from the previous posts.  How to accomplish Coveo Sorting on a typical custom coveo rendering on a sortable field and how to make sorting work when you have Coveo Tabs that list the results out.

Next step, we will learn how we can achieve the same on Java script, there are few situations where this would come in handy.  In my example, what I was trying to do is apply Coveo Sorting on a computed field.  We are all aware that a computed field is in memory and hence we do not have a GUID we could use to work around the active issue which was present on adding default sort order to Custom Coveo rendering.  With out GUID, the sorting will not work on the rendering due to the issue, this would not be a problem if you are using latest version of Coveo, but, in our case we had no option, but, to battle this challenge.

A option that you could achieve sorting via javascript was the key to successfully getting the sort to work on a sortable computed field.

Below are few resources that helped me get started.

https://answers.coveo.com/questions/1658/how-can-i-set-the-default-sort-order-using-the-jav.html

Below is the code snippet that I plugged in my Custom Coveo View which needed the default sort order to be set

CoveoSortComputed.PNG

Do note that you have to set this in function, so, the properties get set properly and applied on XHR request. Mission accomplished.

Now, I covered all the ways that we applied Coveo sorting on different style components. Another cool thing I noticed if it helps some one is that even if your field is of type Date, do not use that option for Default Sort Order. Just use Field as your type and all is working as expected.

Happy Problem Solving people. Enjoy your rest of the weekend! I am actually blogging from SF airport after hogging on organic veggie quesadilla. 😉

Coveo Sorting Part 2

Hi There!

Welcome back again to my three part series of sorting with Coveo.
This time, we look more deeper in to how to make your tabs have a sorted list, below are few easy steps you will need to do make sorting work on Coveo tabs.

  1. Ensure you have a Sort Component added to your presentation.  If you do not do this step, the Default sort order you would enter on Coveo Tab Component would not work and could also lead to exceptions on console.   Read more details about Coveo Sort Component here if curious https://developers.coveo.com/display/public/JsSearch/Sort+Component;jsessionid=6D194F8CCE1AC89DDB1B92C50E446CFF
  2. See screenshot below to see how a Coveo Custom Sort View would look like.
  3. Ensure you plug in the same Default Sort Properties you have defined on your Coveo Custom Tab, which means you should match Sort Order, Field and Order exactly to same values on both your components for the sort to work fine.
 Shown below is the view of Custom Sort View that goes hand in hand with Coveo Tab
SortView
Shown below is the Sort Order Default defition on Coveo Sort View
CoveoSortView
Shown below is the Default Sort order properties on Coveo Custom Tab view. Notice how they are exactly the same, except that on the below we have to use GUID to overcome an existing bug on version of Coveo we are using.
CoveoTabSortOptions
That is it, after this steps, publish your changes and go to the page where you have coveo renderings with tabs, you will now see two things. One, your sort options will be appended to the URL by default and you can also confirm the parameters passed on the Network tab in the XHR request.
If all is well, you now have your sort order working on Coveo Custom tabs.