How to utilize continuous updates with your imported dataset

Integrations are features available to our direct importers from data collection platforms. They enable the user to add new cases that have been directly imported without having to go through an append process.

Continuous updates (integrations) are a special type of tracking studies

There are two approaches to tracking: Align and combine and Continuous updates.

Align and combine

You import new data as a distinct dataset. You then prepare the new dataset (align) so that you can append it (combine) to the target schema (ie: the main tracker).
This is the process described in the Definitive Guide to Tracking.
This is what you need to do if you:

Import via datafiles (e.g., SPSS, CSV, and so on),
Are not using one of the direct importers mentioned above, and
Have different data collection links for each tracking wave.

Continuous updates (integrations)

This is where you pull data continuously into the one dataset using certain direct importers (see above).
It is termed an integration because where Crunch detects new cases in the survey platform, Crunch adds them as new rows in the dataset.
There is no separate explicit alignment process by the user in a separate dataset - the incoming data conform to Crunch schema via a series of rules.
The rules are required because if you change the schema in Crunch (eg: relabel a category), then Crunch needs some basis to decide what to do with the new data.

For example, you have an answer option in a question that is the brand “Coca-Cola”, but you relabel the equivalent variable’s category in Crunch as “Coke”... then Crunch needs some rule to deal with the discrepancy. Should it be “Coca-Cola” or should it be “Coke” because you've made that change in Crunch? The rule defaults to the latter: any new cases that are added have the “Coca-Cola” answer option mapped into the Crunch category “Coke” for that variable.

In the following, we outline what the rules are for continuous updates so you understand what to expect when you make changes in either platform and what limitations exist.

The Rules

How Crunch reconciles changes you make to the dataset

Schema change in Crunch	Outcome	Example
Change variable title/description/notes	The Crunch metadata will persist after the update
Change category labels (names)	The Crunch category label will always be preferred. There are some warnings about this: Adding categories is okay Changing category labels in Crunch is fine as long as there isn’t another category with the previous label. Deleting and then adding the same category label back later is unresolvably ambiguous.	You have “Coca-Cola” as an answer option on a single-response question. You change it to be “Coke” in Crunch After updating, it remains Coke in Crunch and any new responses to “Coca-Cola” by participants, are mapping into “Coke”.
Change the label on a subvariable	The Crunch subvariable label will always be preferred.	You have “Coca-Cola” as an answer option on a multiple response or a grid-style question. You change it to be “Coke” in Crunch After updating, it remains Coke in Crunch and any new responses to “Coca-Cola” by participants, are mapping into the “Coke” subvariable.
Change order of subvariables in an array	The Crunch order of subvariables will persist after update.
Change numeric values	The Crunch value assigned to categories will persist after update.	You have a scale question and you decide to change the values from 5 to 1 to 1 to 5 (ie: reverse the scale in Crunch). The scale will stay as you set it in Crunch (ie: it won’t flip back to the way you don’t want it).
Organizing into (different) folders	The Crunch folder allocation is unaffected by the update.
Any derived variables	All derived variables will compute for the new cases automatically.	Any variable that you create in Crunch (and not in the survey) will be updated - because the Crunch variable refers to other variables that get update. For example, you collapse a 10-point scale into 3 categories. This is a new separate variable. This new variable will update, because the 10-point scale you created it from also updates.
Creating weighting variable	The new cases will have weight value in line with the original the weight definition. In other words, the weighting variable does not automatically readjust to the new sample size. You will need to recreate the weight.	You run a study with n=500, where n=300 are male and n=200 are female. You make a weight with weighting targets where male and female at 50/50 respectively. You then have an additional 50 men and 50 women complete the study. So in total, you now have n=350 male and n=250 female. The new respondents receive the same weight factors as the original n=500, so when you look at the gender variable with the weighting applied, it will NOT show 50/50. You need to delete the weight and recreate it if you want to have 50/50.

How Crunch reconciles changes you make to a survey

Do NOT delete anything — hide it instead. Deleting things (subvariables/rows or categories) will cause the schema to break and thus the integration.

Survey change	Outcome	Example
Categorical or numeric variable added	Will appear in Crunch as a new variable, with missing data historically.	You add in a question about a participant’s political preference. The new question appears in Crunch after the next update and remains thereafter. Any cases before the update have missing data for this question (since they were never asked it)
Subvariable added to an array (categorical array, numeric array, multiple response)	In Crunch, these appear as new subvariables in the relevant array. All respondents will have missing data where the subvariable did not exist previously.	You have an array asking about 3 different cola brands (Coke, Pepsi, Diet Coke). You decide you need to add Pepsi Max to the array. Pepsi Max is visible in Crunch after the export, but has missing data for all cases prior to its inclusion in the survey.
Subvariable hidden from an array (Warning do NOT delete a subvariable)	You cannot delete a variable or subvariable - it will cause the integration with Crunch to break (as the schema has fundamentally changed). If you hide a subvariable in an array, then it will be missing values after the update.	You have an array asking about 3 different cola brands (Coke, Pepsi, Diet Coke). You decide you don’t want to include Diet Coke anymore in the study. You do NOT delete it in survey. You hide the subvariable (row) in the survey using the panel on the right-hand-side. Whatever respondents then complete the study from that point forward, will have missing values on this subvariable. If it’s “unhidden”, then it responses will again flow in as valid (i.e., non-missing). So you can turn subvariables off/on as being missing.
New category added into a single-select question	New category added to variable in Crunch.
Category removed from a single-select	You cannot remove category in the survey - it will cause the integration with Crunch to break (as the schema has fundamentally changed). You can hide the category you don’t want in the survey. Back in Crunch, you have the option to set it as a missing category, if you so wish.	You have a 0-10 rating scale in a categorical array or in a categorical question, along with a “Don’t know” option. You decide you no longer wish to have the Don’t Know category as an option for the respondents. You do NOT delete the category in the survey. You hide it using the panel on the right-hand-side. You cannot delete the category in Crunch - you can set it as a missing category if you wish.
Category added to a categorical array	New category added to variable in Crunch.	You add a “Don’t know” answer option in a matrix scale question in the survey. The “Don’t know” option appears in the categorical array in Crunch.
Category name changed	No category label change in Crunch	You change the answer option from “US” to “USA” in the survey. This makes no difference in Crunch - the category will be as you had it before the update. You may need to update Crunch if you want to reflect the survey.
Category removed from a categorical array	You cannot remove a category in the survey - it will cause the integration with Crunch to break (as the schema has fundamentally changed). You can hide the category you don’t want in the survey. Back in Crunch, you have the option to set it as a missing category, if you so wish.	You remove “Don’t know” as an answer option in a matrix scale question in the survey. Crunch maintains the “Don’t know” category - but any new cases won’t have any data in that category.

Help Center