What is Inventory?


An inventory item is any piece of content that we could potentially show in recommendations. In general, anything that can have a URL* can be an inventory item. For instance:

  • On a typical news or viral content website, the inventory items are articles.
  • On a video website, the inventory items are videos.
  • On a retail website, the inventory items are the products being sold.


We’ll use the term "field" for each piece of information or metadata about a given item. For instance, the typical fields for an article inventory item include title, URL, image, and description. 


By default, LiftIgniter collects all Open Graph tags to construct the metadata for your pages. This includes tags that begin with og: as well as some article:, music:, video: and book: tags. You can get detailed documentation about standard Open Graph tags at the Open Graph protocol website.


*In some cases, particularly e-commerce, you may want or need to use an actual ID as the unique identifier for an item, instead of a URL. To enable the use of an ID for your account, please contact Support!


Setting up your Inventory


Prior to provisioning your account, we identify what "inventory item" means in the context of your website, and set up our scraping to collect inventory information only for those types of items. We generally set up a preliminary filter based on the pattern of the URL. For safety, the Javascript beacon will not begin collecting inventory until that preliminary filter is activated by our Engineers.


Once you install our Javascript, we start scraping any pages that match the URL filter. Every time a user visits a page on your website that meets that criteria, we collect all the fields from that page and send it to our servers. If the item is not currently in our inventory, we add it. If it already exists, we will replace the old version with the new version. This means that if you remove metadata from the item, that metadata will also be removed from our inventory.


Any item that is in the Inventory has a chance to appear in LiftIgniter recommendations. You can, however, block specific items from appearing as recommendations.


Preventing duplicate items


There are a few common ways that duplicate items may appear in our system. Because the URL is the unique identifier for all items, any changes or differences in a URL for a single item can result in duplicates.


Desktop vs Mobile Versions

One common cause of duplicate items is using separate URLs for mobile and desktop versions - typically www. and m. subdomains. To prevent this, make sure that the canonical URL is consistently one version or the other. You could also consider passing the "url" field in the LiftIgniter JSON object to ensure we only receive a single URL. 


HTTP vs HTTPS


The protocol of a URL can also create duplicates, particularly if your site does not automatically redirect to HTTPS. In general, we recommend using HTTPS as the canonical version of the URL for sites that have a valid SSL certificate.


If you are currently using HTTP and would like to switch to HTTPS, please let us know! We can perform a one-time migration of data in order to switch our inventory and activity logs to the new HTTPS protocol - this ensures that there is no loss in recommendation quality. Please note that this process does take some time to complete - notify us early and we will be happy to coordinate with your own change-over.


Changing the URL After Publication


While uncommon, sometimes the URL is published with a typo, or doesn't match the links that were already sent out. It happens! If the old version of the URL has already been collected into our Inventory, it will need to be removed to prevent both the old and the new version potentially being recommended side-by-side (which can happen particularly with trending content). To remove it, you will need to use a tool like Postman to send a DELETE command via our API. 


How often does the Inventory update?


Our inventory is kept up-to-date by users visiting the content - this triggers our Javascript beacon to fire and scrape the page again. As a result, you'll want to keep in mind the following conditions:

  • We can only detect pages that have been visited at least once by a user while our Javascript was installed.
  • The inventory collection function only fires after the page DOM has completely finished loading - if your pages take longer than a few seconds to load all content, users may leave before the inventory function has a chance to fire and collect the page data.
  • We update fields for a page only when it is visited, and it gets deleted if nobody has accessed it for more than 30 days (we can remove the TTL depending on your use case).
  • If a page is deleted from your CMS, you'll need to remove it from our inventory using the DELETE command for the /inventory API.


Inventory updates via Javascript do take some time to process. In general, you should see new items or updates appearing in the inventory within a few minutes of publication, but it can take up to several hours due to server maintenance, back-end updates, and other factors. If you are not seeing a new item appearing in your inventory and are concerned, please reach out to Support and we'll be happy to help.


Consider using the API for time-sensitive, real-time updates


The almost-real-time inventory update is sufficient for most of our customers. However, if you have more advanced needs, such as rapidly changing content, or a large amount of older content that gets a very small number of views, or want to include sensitive page metadata that should not be publicly visible or scrape-able by other systems, we recommend using our inventory API instead. This allows you to insert, update, and remove items directly from our servers (no processing time required).