How to utilize continuous updates with your imported dataset
Integrations are features available to our direct importers from data collection platforms. They enable the user to add new cases that have been directly imported without having to go through an append process.
Continuous updates (integrations) are a special type of tracking studies
There are two approaches to tracking: Align and combine and Continuous updates.
Align and combine
- You import new data as a distinct dataset. You then prepare the new dataset (align) so that you can append it (combine) to the target schema (ie: the main tracker).
- This is the process described in the Definitive Guide to Tracking.
- This is what you need to do if you:
- Import via datafiles (e.g., SPSS, CSV, and so on),
- Are not using one of the direct importers mentioned above, and
- Have different data collection links for each tracking wave.
Continuous updates (integrations)
- This is where you pull data continuously into the one dataset using certain direct importers (see above).
- It is termed an integration because where Crunch detects new cases in the survey platform, Crunch adds them as new rows in the dataset.
- There is no separate explicit alignment process by the user in a separate dataset - the incoming data conform to Crunch schema via a series of rules.
- The rules are required because if you change the schema in Crunch (eg: relabel a category), then Crunch needs some basis to decide what to do with the new data.
- For example, you have an answer option in a question that is the brand “Coca-Cola”, but you relabel the equivalent variable’s category in Crunch as “Coke”... then Crunch needs some rule to deal with the discrepancy. Should it be “Coca-Cola” or should it be “Coke” because you've made that change in Crunch? The rule defaults to the latter: any new cases that are added have the “Coca-Cola” answer option mapped into the Crunch category “Coke” for that variable.
In the following, we outline what the rules are for continuous updates so you understand what to expect when you make changes in either platform and what limitations exist.
How Crunch reconciles changes you make to the dataset
|Schema change in Crunch||Outcome||Example|
|Change variable title/description/notes||The Crunch metadata will persist after the update|
|Change category labels (names)||
|Change the label on a subvariable||The Crunch subvariable label will always be preferred.||
|Change order of subvariables in an array||The Crunch order of subvariables will persist after update.|
|Change numeric values||The Crunch value assigned to categories will persist after update.||You have a scale question and you decide to change the values from 5 to 1 to 1 to 5 (ie: reverse the scale in Crunch). The scale will stay as you set it in Crunch (ie: it won’t flip back to the way you don’t want it).|
|Organizing into (different) folders||The Crunch folder allocation is unaffected by the update.|
|Any derived variables||All derived variables will compute for the new cases automatically.||
|Creating weighting variable||
How Crunch reconciles changes you make to a survey
Do NOT delete anything — hide it instead. Deleting things (subvariables/rows or categories) will cause the schema to break and thus the integration.
|Categorical or numeric variable added||Will appear in Crunch as a new variable, with missing data historically.||
|Subvariable added to an array (categorical array, numeric array, multiple response)||
|Subvariable hidden from an array
(Warning do NOT delete a subvariable)
|New category added into a single-select question||New category added to variable in Crunch.|
|Category removed from a single-select||
|Category added to a categorical array||New category added to variable in Crunch.||
|Category name changed||No category label change in Crunch||
|Category removed from a categorical array||