About This Release
Released - December 2, 2021
This version of LS Insight supports versions 8.4 and later of LS Central On-premises and later versions of LS Central In-cloud (SaaS). Setup of LS Insight for LS Central In-cloud (SaaS) needs some additional setup from the standard LS Insight onboarding process which is explained in detail in the onboarding documentation.
In this version we have added two pipelines to the Azure Data Factory to simplify the setup and maintenance of LS Insight. The first one is the Initial load pipeline and the second one is the Factory reset.
We have also added a section on how LS Insight can be extended by additional data from LS Central and LS Central extensions and with third party data from any source.
Note: Updates for version 2021.3 are provided in a special update package. If you have already set up version 2021.3, you can update to 2021.4 without creating a new instance of LS Insight. See update guidelines for more information. Please note that if you have customizations in your LS Insight instance, you need to make sure that the updates do not affect these customizations before you start. If you are running LS Insight against LS Central in SaaS, you will not be able to update LS Insight but need to set up a new instance of LS Insight.
The following is a list of changes and key features in LS Insight version 2021.4.
New or Enhanced Features
Setup
Onboarding
Onboarding documentation has been changed and now explains how the Initial load and Factory reset pipelines should be used.
Documentation
Extending LS Insight
A section has been added on how LS Insight can be extended by additional data from LS Central and LS Central extensions and with third party data from any source. The extension page contains high-level information on extending the LS Insight product and you can download more detailed extension guidelines.
Data Warehouse
Database
Tables & Stored procedures
dItem - Slowly changing attributes added
Item dimension now includes Attributes that are SCD type 2 (slowly changing). Season, category, product group, and division will now record change history.
Season Is SCD2, ProductGroup is SCD1 but [scd2-ProductGroup] is a SCD2 version for the same attribute. Same applies for the following pares: VendorNo & [scd2-VendorNo], Category & [scd2-Category], Division & [scd2-Division]
SCD1 will always include the current value in all versions of the selected Item.
SCD2 will keep the change history with valid from and to dates. Fact tables will connect to the correct version using the valid dates.
Example:
Initial state of Item 10000 has no season and is valid from 1984-05-12 to 2172-02-23.
If a season is changed, a new line will be added with the current version. The ValidTo date for the previous current version will be set to today – 2 days and ValidFrom for the new version is set to today -1 day.
If the Description (SCD1) is changed, all versions will be changed. And if SCD2 attributes are changed in the same process, a new version will be added:
Data type changes
Data types in the following tables and columns have been updated to fit the max possible value from LS Central:
- RowID in dCountryRegion is now bigint
- Country in dHotelReservations is now nvarchar(100)
- Description in dHotelRooms is now nvarchar(2000)
- Description in dItemVariant is now nvarchar(100)
- LocationName and ReportingStoreName in dLocation are now nvarchar(100)
- Description in dReturnReason is now nvarchar(100).
dDate
A new date dimension table has been added. It is dynamically created based on min. and max. sales dates and is updated on a schedule, but is currently not used in the report templates.
Stored procedures
Members are assigned to age band in the Member dimension.
dimMergedDate - this procedure has been added and maintains a date table. The date table is currently not used in our reports, but this was done to prepare for allowing users to use our sales report in direct mode with aggregation, if needed.
Pipelines
PopulateQueryBase
Generate query base pipeline - triggers cannot start an activity while previous triggers are running.
Company staging table has been removed. Company information has been added to the LSInsight$Companies table and that is the information used to populate the dCompany dimensions table.
All staging tables
The last column in all staging tables now stores the trigger run GUID from the Azure Data Factory pipeline activity that inserted the record.
In an initial load the staging pipeline now uses the min timestamp from the source instead of assigning a "0" as the first record to retrieve.
Initial load - new
This new pipeline executes the Populate Query Base and Scheduled Run pipelines and it should only be manually triggered during the initial setup of LS Insight.
Reset LS Insight DW - new
This pipeline runs a stored procedure on the database that deletes all retail data from the database, without deleting any of the necessary meta data that is included when the database is created from the template. This pipeline should always be run as a part of the Factory reset pipeline.
Factory reset - new
This new pipeline first executes the Reset LS Insight DW pipeline to clean all retail data from the data warehouse. Then it runs the Initial load pipeline to fully populate LS Insight with data from LS Central.
Tip: If you are experiencing some data issues in LS Insight that might be caused by importing data to LS Central from an external system, or if you have at some point changed the LS Central source for LS Insight, it is good to run Factory reset to make sure that the issues you are experiencing are not caused by data mismatch between LS Central and LS Insight.
Reports
LS Insight Sales
No changes.
LS Insight Inventory
New inventory template because of bug fix. See below.
LS Insight Hotels
No changes.
Bug fixes
Incorrrect FiscalYearStart in dCompany table. Fixed by adding a Where clause with NewFIscalYear = 1 to the predimMergedCompany stored procedure.
Fixed issues in SP for Inventory related to Transfers and open Documents and on the OY calculations in the Inventory report template. New template has been provided that fixes this.
Issues with Populate query base pipeline caused by rouge extensions has been fixed. Now extension data is only loaded from tables that are present in the LSInsight$PublisherAffixReg table.