September 14th, 2021

Colectica Portal


  • Allow and show weighting of summary statistics, Added variablestatistics weight tab to portal view

  • Controlled vocabularies for CodeValue fields

  • Portal basket: allow re-ordering items

  • Portal /search should use separate Elasticsearch method when Elasticsearch is present, or a DBsearch method when not present

  • File uploads

  • New route, /searchdb , to always run the Repository search

  • Use new elastic index style, Create search UI from addin facets

  • Show NCube information on PhysicalInstance page

  • Elastic indexer task runner (net5)

  • allow setting an Identity Password Policy in the appsettings

  • Extra translations for account registration page

  • add replication controller to portal

  • add workflow configuration for Portal appsettings

  • Custom addins for Portal using Razor class libraries, loaded from the parts/ directory as AssemblyParts

  • Allow customization of the cookie warning text

  • More translations

  • webdav and disk backend for uploads and downloads

  • file metadata retrieval for file replications

  • file upload and download for file replications

  • display download links for other materials and external instruments


  • For StudyUnits, when the same DOI is present in both DublinCore.Identifiers and in UserIDs, only show it once

  • don’t run a search query from the search page for all item types if no search query model is present

  • sorting by id when running a search with no terms based just on item types

  • allow openid connect tokens when no email profile is present

  • switch onvalidate openid connect to async

  • additional problemdetail api responses for permission failures and async calls, better api disabled 404

  • clean up api controller base, no extra onactivating needed

  • clean up repository caller, remove unused params

  • In concise concordance view, show the count of variables, not the count of PhysicalInstances

  • Move UI search facets from sidebar to main content area

  • Normal form submission instead of JS

  • Concordance updates for CLOSER project

  • Explore view: support multiple ConVArGroups for a Concept

  • Show correct count of variables in concise view

  • Remove unused JS

  • Concordance navigation fixes when the ConceptualVariableGroup layer is present. Use query parameters instead of 8+ unlabled route parts.

  • Option to default to concise concordance view

  • Restore the type-change indicator when different RepresentedVariables are used for the same ConVar

  • In concise concordance, show blank cells instead of “0 variables”

  • Don’t show the entire Description/Abstract (summary) in search results, as they can be very long

  • Portal item pages: don’t show properties for plain strings that have no content

  • Portal item pages with DublinCore Contributors: show better property name (creator, contrib, publisher). Don’t show big <h4>

  • Portal concordance, concise view: show the checkboxes to toggle columns on or off

  • Portal concordance, verbose view: Fix checkboxes, to make sure they always toggle the correct columns and spans

  • When deciding whether to show a concordance table or a list of ConVarGroups, do not consider ConVarGroups with 0 items in them.

  • only use elastic when Enabled=true in the appsettings

  • Don’t show concordance table on Concept page. Instead, show lists of related RepVarGroups, ConVarGroups, ConVars

  • option to not validate openid issuers, Microsoft uses a common issuer for all AzureADs when allowed

  • When only the Description of an Organization is present, show it

  • Show Universes of Variables

  • Show the ConceptScheme Description on the Explore page when nothing is selected

  • Don’t cache topic hierarchies generated from ConceptSchemes for the explore view

  • More prominent comparability note display

  • For data extracts, get identity variable information from repository, not Elastic

  • Show UnitType on ConVar and RepVar pages

  • CSS classes on <body> tags for customizations

  • Get the correctly localized item names for the AppearsWithin tree

  • Don’t show Create Account link when using OpenID

  • Don’t show Manage link on profile page unless it’s local identity

  • Tooltip for type icons in the variables TableLocation

  • raised default maxAllowedContentLength in web configs

  • Only show VariableGroups that reference variables when the latest version of the variable group has the reference

  • allow a way to disable the facet box for the series

  • Show IntendedFrequency

  • Show the basket’s title on the cover page of a Basket PDF

  • Concordance tables can be organized by VariableGroups instead of PhysicalInstances


  • Correct code and numeric images

  • Handle Elastic Search not being configured on status page

  • Search direct relationships, not full sets, when finding classification items

  • Portal StudyUnit page: show actual AnalysisUnit content, not types

  • Show a nicer error message when something goes wrong creating the concordance view.

  • Populate NCubes correctly for Portal display on PhysicalInstance page

  • Fix error in Archive page, when an Organization was present

  • Show more decimal places when they are present

  • Get basket to work in concordance tables for harmonization groups

  • Fix issue with OpenID Basket page redirect loop

  • 404 when an item is not found in repository, instead of showing error

  • show a nicer error message when elastic data downloads fail

  • don’t display breadcrumbs on search page before Phase 2 elastic indexing of the item is complete

  • When a Variable links to a GenerationInstruction (via CustomField), the link on Portal has no text. Use the description.

Colectica Designer


  • Qualtrics to DDI importer for .qsf files

  • SKOS import into ConceptSchemes or CodeLists

  • Controlled vocabulary, site wide specification for specific fields, cv resolution

  • Allow drag and drop to re-order multiple code values

  • Upload files to Repository

  • Citation enhancements: Allow multiple contributors, publishers, and creators with detailed properties

  • For difference viewer, show an HTML summary that points out differences within strings

  • Support semicolon delimited files for Delimited Import

  • Designer command to ingest cross-study concordance (concording ConceptualVariables into HarmonizationGroups)

  • Added context menu in NavigatorView to allow for entire repository to be browsed by DdiItemType

  • Command to generate a DDI 3.3 DDIInstance

  • Allow and show weighting of summary statistics

  • Calculate weighted summary statistics

  • Added grouping in designer based on ismissing statistics Value

  • Enable both v6 and v7 replication requests in Designer 6.2

  • Editors for PhysicalInstance.VariableGroups

  • Allow deprecating a single item, instead of the entire set


  • Update classification task mode to work with DDI 3.3 item types, instead of Copenhagen Mapping types

  • Detect repository configuration changes for repositories that exist on launch

  • For text variables (including all CSV values), detect min and max of “numeric” data correctly, not alphabetically

  • Only include valid values in Median/Quartiles when computing from SAS

  • More efficient population for concordance ingest command

  • Report convar names that are duplicated in concordance ingest sheets

  • Sortable DataGrid for concordance messages, to allow sorting errors vs non-error messages

  • add replication commands to repository client base, rest

  • update replicate command in Designer to use Portal api and repository client

  • remove WorkflowServerManager, workflow server config from backstage

  • Concordance ingest: Report variables that are listed on multiple rows, and ignore second occurrences, Report progress

  • Upgrade and consolidate references to HtmlAgilityPack (dependency of dotNetRDF)

  • Designer History view: use Best not Current when displaying messages

  • Designer editor: only show a single line of the item’s title, in case the title includes line breaks

  • Designer Links view: show Concepts related to Concepts, Do not show same item in list of item links

  • Concordance ingest: add newly created ConVars to allConVars, in case they are listed twice in the spreadsheet

  • Designer: only show the first line of a study’s abstract in the search results

  • When creating a PhysicalInstance from an Instrument, copy the Question Label into the Variable Label, instead of the QuestionText

  • On concordance import, Locate top-level container ConVarGroup correctly when only one exists

  • Refresh checkout more nicely after running concordance import command

  • Option not to remove groups when synchronizing a group structure to a ConceptScheme

  • For create-dataset-from-instrument command, create variables for choice grids

  • Show version responsibility and rationale in difference viewer

  • Allow UnitType on RepresentedVariable

  • Don’t allow making new Universe references on ConVars and RepVars, only UnitType

  • Allow either “Concept” and “Topic” to be used as column header for concept import sheets

  • Allow commands to indicate that the checkout should be updated

  • Assign correct weight variable for text variables

  • Bulk ingest: when no data file exists but a metadata input sheet exists, create a PhysicalInstance based on the input sheet

  • When calculating summary statistics for coded text variables, count frequencies for values in the variable’s MissingValues list

  • When determining the Minimum and Maximum for text variables that actually contain numbers, do not ignore the first value

  • When determining whether a variable is numeric or text based on the data file format, do this for both valid and invalid values

  • Don’t allow removing Categories from under CodeLists using the navigator. It removes the Category but not the Code, which is confusing.

  • Don’t show NCube previews when there are too many dimensions

  • Links side panel in Designer needs a vertical scroll bar

  • Calculate summary statistics: prompt user for file on disk to use

  • editor for a Variable Representation’s ClassificationLevel. Similar to Additivity and Role.

  • instrument editor: double click a Summary item or Questions view item to navigate to that item

  • CollectionEvent listbox: show content from mode of collection or other fields when no date information is present

  • Show basic textbox for CodeValueControl when no CV is configured

  • Instrument comparer: when a code list changed but the code list has no name, show the question’s name

  • Show active repository in synchronize dialog

  • Index text for GenerationInstruction and GeneralInstruction

  • Show full URI in repository selector

  • When calculating summary statistics on all datasets in a Group, allow selecting unweighted stats

  • When calculating summary stats for all datasets in a group. don’t require user to specify a file. Use the one on the PhysicalInstance


  • When settings cannot be read, load the default settings

  • Re-save settings file after an invalid one is loaded, to make sure it is okay for future access

  • Settings initialization and error handling should happen before using a setting the first time

  • Use Best not Current for names, labels etc in source code generation

  • Handle TimeSpan from SAS

  • Concordance Import, When matching variable names from a spreadsheet, Trim() the spreadsheet contents

  • Blaise import: support SPECIALANSWERSETS

  • Designer difference viewer: fix crash when saving CSV of changes

  • ScrollViewers for all views under the Archive control

  • Show more decimal places when they are present

  • Remove ScrollViewer from wizard view. It prevented the DataGrid from virtualizing.

  • items from repository should never be provisional, ignore this flag if set by a repository

  • Fix bug where editing references within a CollectionEditor changed the wrong item

  • Find Items with Updates: don’t use remote client for items that are not on the remote repository

Colectica Repository


  • Repository transaction commit hooks

  • New propagate transaction commit type that can SetLatestToTrue

  • Added a UpdateRepositorySetLatestChildrenOnLatestItemsAndPropagateVersions transaction

    • This type of transaction is run without any user submitted items

    • Each latest item in the Repository, or in the specified set, is checked for children that are not latest

    • Older child references are replaced with latest child references

    • Propagation of updated items is done in the same manner as CommitAsLatestWithLatestChildrenAndPropagateVersions

    • Scoped Sets are currently not supported with this repository update type

  • api for DDI XML fragment and DDI FragmentInstance sets

  • In Elastic, Index abstract, purpose, contributors, creators, publishers, subjects, keywords, related organizations (as extraText)

  • For CVs, index plain values and multilingual matched category information in separate fields

  • Enable file uploads

  • Calculate SHA256 and MD5 of deposited files


  • use problem details and proper http status codes for known repository errors

  • use one combined query for repository statistics

  • In postgresql search, remove secondary sorting by alpha for non alphabetical sorts

  • don’t calculate metadata rank unless RankResults is true or sorting by metadata rank

  • Increase timeout of database transaction scopes

  • Ensure StudyAbstract has text indexed

  • count outdated parents as user submitted, so a count of outdated can be determined before version propagation

  • don’t include duplicates in update repository to latest children

  • add text extract for individuals

  • use a DB transaction for sqlite in registeritems instead of a general transactionscope

  • explicitly set all latest fields in sql server upgrade script


  • Prefix search on postgresql

  • only allow one prefix in a search term on postgresql

  • always set format to RepositoryFormats.Ddi33 when rewriting repository items in transactions

  • Index actual values of subjects and keywords

  • handle non concrete items in transaction propagation

  • cleanup text extracts, no duplicate names and labels

  • script to reset sql server latest flag

Colectica SDK


  • nuget targets for both net472 and netstandard2

  • Map more content from DDI 2 to DDI 3

  • DDI 2 reading: support multiple fileDscr elements, creating multiple PhysicalInstances

  • nuget package for Colectica.Reporting

  • Upgraded HtmlAgilityPack

  • TimeoutInSeconds property, used for REST clients

  • For CategoryStatistics, allow both Frequency and WeightedFrequency

  • For DDI 2, import more summary statistics, including weighted frequencies

  • Give a label to Instruments that are created during DDI 2 Import

  • Add GetRepositoryStatistics to RepositoryClientBase and its implementations

  • Add methods to detect DDI format from XML content or streams, in addition to files

  • item indexer pool and interface

  • DDI Item indexer addin support

  • flag to switch off change tracking with static VersionableBase.DisableChangeTracking

  • add a SucceededBy property to Iversionable

  • Make the selected remote repository available to addins

  • add VariableGroups to PhysicalInstance


  • In DDI 3.3, do not store OtherMaterial CustomFields on the parent item

  • DDI 2 reader: for varGrps with no variables, don’t try to locate a PhysicalInstance based on the variables

  • DDI 2 mapping updates

  • DDI 2 look for a fileDscr ID in more places.

  • Remove Repository dependency from ViewModel assembly

  • Move LocalCheckout to Model assembly to enable that

  • Use RepositoryClientBase in ViewModel project. Pass in LocalRepositoryClient from callers as needed.

  • In DDI 2 import, support date ranges with ‘YYYY’ for Year, instead of full dates

  • Safe handling for URI creation in DDI 2 import

  • Treat two multilingual strings with HasValue == false as equal

  • DDI 2 import multiple OtherMaterials when they exist

  • DDI 2 import Option to always create a wrapper sequence for questions that have pre/post text

  • When ingesting DDI 2, use the designated language, not .Current

  • 3.1 Questacy import, gracefully handle incorrectly created identifiers and references


  • Do not add DataCollection to StudyUnit twice in DDI 2 import

  • DDI 2 import: For year-only dates, set the DDI 3 Date’s precision correctly

  • add UnitType to GetChildren, RemoveChild, ReplaceChild in RepresentedVariable

Colectica Questionnaires and Blaise Colectica Questionnaires


  • History view and version comparison in Questionnaires

  • In Open window, show instruments in alphabetical order

  • When looking for version state, only use the first configured repository, not all. Consider this the “active” repository

  • Show Instructions in preview cards

  • Edit name and label for display logic

  • For numbers, use validated text boxes, instead of the numeric-up-down spinner controls

  • Allow adding multiple existing items at once

  • Load IVersionableCommands that work against Instruments

  • checks when adding existing items

Colectica for Excel


  • handle datetime variables from SAS in variable statistics computations


  • skip tagged missing values in SAS code list when the list has duplicate entries for one value


  • Microsoft re-signed the old vc9 installer, new certificate added in wix remote payloads

  • Microsoft re-signed the net 472 installer, new certificate in wix remote payloads

Colectica Report Generation


  • Translations for built in reporting strings


  • Show up to two decimal points in percentages

  • In PDFs, show “0” as a frequency, instead of blank

  • For PDF output for variable statistics, for numbers that have more than a few zeros after the decimal separator, show in scientific notation


  • LogicalProduct reports: include all child items

  • Don’t add code lists twice under a variable that does not have summary statistics

Colectica Workflow


  • New workflow website based on netcore

  • New workflow service task runner based on netcore

  • workflow replication of files from source to target repository

  • enable forward headers

  • During workflow approval process, let user choose which files to replicate

  • Show item identifiers as tooltips in the process list

  • Configurable command timeout

  • Show all languages of item labels/names in Workflow Web


  • separate Portal database from portal website project

  • load replication repositories by configuration

  • Allow repository to take a database connection

  • Allow workflow replication db to take a database connection

  • create task runner to process workflow replication events

  • new workflow website api to handle replication requests from repositories

  • re-written DataAccess, use passed in connection strings and providers, remove oracle

  • new database strings connection class

  • ToIdentifier for identifierinrequest

  • Remove old workflow web

  • add estonian to old cache builder

  • calculate preview db path better


  • only VACUUM when isNpgsql

Colectica Toolkit


  • Default to DDI 3.3 for repository import

Colectica Datasets


  • Copy de-DE translations for Datasets installer