6.2¶
6.2.8400¶
September 14th, 2021
Colectica Portal¶
Added¶
Allow and show weighting of summary statistics, Added variablestatistics weight tab to portal view
Controlled vocabularies for CodeValue fields
Portal basket: allow re-ordering items
Portal /search should use separate Elasticsearch method when Elasticsearch is present, or a DBsearch method when not present
File uploads
New route, /searchdb , to always run the Repository search
Use new elastic index style, Create search UI from addin facets
Show NCube information on PhysicalInstance page
Elastic indexer task runner (net5)
allow setting an Identity Password Policy in the appsettings
Extra translations for account registration page
add replication controller to portal
add workflow configuration for Portal appsettings
Custom addins for Portal using Razor class libraries, loaded from the parts/ directory as AssemblyParts
Allow customization of the cookie warning text
More translations
webdav and disk backend for uploads and downloads
file metadata retrieval for file replications
file upload and download for file replications
display download links for other materials and external instruments
Changed¶
For StudyUnits, when the same DOI is present in both DublinCore.Identifiers and in UserIDs, only show it once
don’t run a search query from the search page for all item types if no search query model is present
sorting by id when running a search with no terms based just on item types
allow openid connect tokens when no email profile is present
switch onvalidate openid connect to async
additional problemdetail api responses for permission failures and async calls, better api disabled 404
clean up api controller base, no extra onactivating needed
clean up repository caller, remove unused params
In concise concordance view, show the count of variables, not the count of PhysicalInstances
Move UI search facets from sidebar to main content area
Normal form submission instead of JS
Concordance updates for CLOSER project
Explore view: support multiple ConVArGroups for a Concept
Show correct count of variables in concise view
Remove unused JS
Concordance navigation fixes when the ConceptualVariableGroup layer is present. Use query parameters instead of 8+ unlabled route parts.
Option to default to concise concordance view
Restore the type-change indicator when different RepresentedVariables are used for the same ConVar
In concise concordance, show blank cells instead of “0 variables”
Don’t show the entire Description/Abstract (summary) in search results, as they can be very long
Portal item pages: don’t show properties for plain strings that have no content
Portal item pages with DublinCore Contributors: show better property name (creator, contrib, publisher). Don’t show big <h4>
Portal concordance, concise view: show the checkboxes to toggle columns on or off
Portal concordance, verbose view: Fix checkboxes, to make sure they always toggle the correct columns and spans
When deciding whether to show a concordance table or a list of ConVarGroups, do not consider ConVarGroups with 0 items in them.
only use elastic when Enabled=true in the appsettings
Don’t show concordance table on Concept page. Instead, show lists of related RepVarGroups, ConVarGroups, ConVars
option to not validate openid issuers, Microsoft uses a common issuer for all AzureADs when allowed
When only the Description of an Organization is present, show it
Show Universes of Variables
Show the ConceptScheme Description on the Explore page when nothing is selected
Don’t cache topic hierarchies generated from ConceptSchemes for the explore view
More prominent comparability note display
For data extracts, get identity variable information from repository, not Elastic
Show UnitType on ConVar and RepVar pages
CSS classes on <body> tags for customizations
Get the correctly localized item names for the AppearsWithin tree
Don’t show Create Account link when using OpenID
Don’t show Manage link on profile page unless it’s local identity
Tooltip for type icons in the variables TableLocation
raised default maxAllowedContentLength in web configs
Only show VariableGroups that reference variables when the latest version of the variable group has the reference
allow a way to disable the facet box for the series
Show IntendedFrequency
Show the basket’s title on the cover page of a Basket PDF
Concordance tables can be organized by VariableGroups instead of PhysicalInstances
Fixed¶
Correct code and numeric images
Handle Elastic Search not being configured on status page
Search direct relationships, not full sets, when finding classification items
Portal StudyUnit page: show actual AnalysisUnit content, not types
Show a nicer error message when something goes wrong creating the concordance view.
Populate NCubes correctly for Portal display on PhysicalInstance page
Fix error in Archive page, when an Organization was present
Show more decimal places when they are present
Get basket to work in concordance tables for harmonization groups
Fix issue with OpenID Basket page redirect loop
404 when an item is not found in repository, instead of showing error
show a nicer error message when elastic data downloads fail
don’t display breadcrumbs on search page before Phase 2 elastic indexing of the item is complete
When a Variable links to a GenerationInstruction (via CustomField), the link on Portal has no text. Use the description.
Colectica Designer¶
Added¶
Qualtrics to DDI importer for .qsf files
SKOS import into ConceptSchemes or CodeLists
Controlled vocabulary, site wide specification for specific fields, cv resolution
Allow drag and drop to re-order multiple code values
Upload files to Repository
Citation enhancements: Allow multiple contributors, publishers, and creators with detailed properties
For difference viewer, show an HTML summary that points out differences within strings
Support semicolon delimited files for Delimited Import
Designer command to ingest cross-study concordance (concording ConceptualVariables into HarmonizationGroups)
Added context menu in NavigatorView to allow for entire repository to be browsed by DdiItemType
Command to generate a DDI 3.3 DDIInstance
Allow and show weighting of summary statistics
Calculate weighted summary statistics
Added grouping in designer based on ismissing statistics Value
Enable both v6 and v7 replication requests in Designer 6.2
Editors for PhysicalInstance.VariableGroups
Allow deprecating a single item, instead of the entire set
Changed¶
Update classification task mode to work with DDI 3.3 item types, instead of Copenhagen Mapping types
Detect repository configuration changes for repositories that exist on launch
For text variables (including all CSV values), detect min and max of “numeric” data correctly, not alphabetically
Only include valid values in Median/Quartiles when computing from SAS
More efficient population for concordance ingest command
Report convar names that are duplicated in concordance ingest sheets
Sortable DataGrid for concordance messages, to allow sorting errors vs non-error messages
add replication commands to repository client base, rest
update replicate command in Designer to use Portal api and repository client
remove WorkflowServerManager, workflow server config from backstage
Concordance ingest: Report variables that are listed on multiple rows, and ignore second occurrences, Report progress
Upgrade and consolidate references to HtmlAgilityPack (dependency of dotNetRDF)
Designer History view: use Best not Current when displaying messages
Designer editor: only show a single line of the item’s title, in case the title includes line breaks
Designer Links view: show Concepts related to Concepts, Do not show same item in list of item links
Concordance ingest: add newly created ConVars to allConVars, in case they are listed twice in the spreadsheet
Designer: only show the first line of a study’s abstract in the search results
When creating a PhysicalInstance from an Instrument, copy the Question Label into the Variable Label, instead of the QuestionText
On concordance import, Locate top-level container ConVarGroup correctly when only one exists
Refresh checkout more nicely after running concordance import command
Option not to remove groups when synchronizing a group structure to a ConceptScheme
For create-dataset-from-instrument command, create variables for choice grids
Show version responsibility and rationale in difference viewer
Allow UnitType on RepresentedVariable
Don’t allow making new Universe references on ConVars and RepVars, only UnitType
Allow either “Concept” and “Topic” to be used as column header for concept import sheets
Allow commands to indicate that the checkout should be updated
Assign correct weight variable for text variables
Bulk ingest: when no data file exists but a metadata input sheet exists, create a PhysicalInstance based on the input sheet
When calculating summary statistics for coded text variables, count frequencies for values in the variable’s MissingValues list
When determining the Minimum and Maximum for text variables that actually contain numbers, do not ignore the first value
When determining whether a variable is numeric or text based on the data file format, do this for both valid and invalid values
Don’t allow removing Categories from under CodeLists using the navigator. It removes the Category but not the Code, which is confusing.
Don’t show NCube previews when there are too many dimensions
Links side panel in Designer needs a vertical scroll bar
Calculate summary statistics: prompt user for file on disk to use
editor for a Variable Representation’s ClassificationLevel. Similar to Additivity and Role.
instrument editor: double click a Summary item or Questions view item to navigate to that item
CollectionEvent listbox: show content from mode of collection or other fields when no date information is present
Show basic textbox for CodeValueControl when no CV is configured
Instrument comparer: when a code list changed but the code list has no name, show the question’s name
Show active repository in synchronize dialog
Index text for GenerationInstruction and GeneralInstruction
Show full URI in repository selector
When calculating summary statistics on all datasets in a Group, allow selecting unweighted stats
When calculating summary stats for all datasets in a group. don’t require user to specify a file. Use the one on the PhysicalInstance
Fixed¶
When settings cannot be read, load the default settings
Re-save settings file after an invalid one is loaded, to make sure it is okay for future access
Settings initialization and error handling should happen before using a setting the first time
Use Best not Current for names, labels etc in source code generation
Handle TimeSpan from SAS
Concordance Import, When matching variable names from a spreadsheet, Trim() the spreadsheet contents
Blaise import: support SPECIALANSWERSETS
Designer difference viewer: fix crash when saving CSV of changes
ScrollViewers for all views under the Archive control
Show more decimal places when they are present
Remove ScrollViewer from wizard view. It prevented the DataGrid from virtualizing.
items from repository should never be provisional, ignore this flag if set by a repository
Fix bug where editing references within a CollectionEditor changed the wrong item
Find Items with Updates: don’t use remote client for items that are not on the remote repository
Colectica Repository¶
Added¶
Repository transaction commit hooks
New propagate transaction commit type that can SetLatestToTrue
Added a UpdateRepositorySetLatestChildrenOnLatestItemsAndPropagateVersions transaction
This type of transaction is run without any user submitted items
Each latest item in the Repository, or in the specified set, is checked for children that are not latest
Older child references are replaced with latest child references
Propagation of updated items is done in the same manner as CommitAsLatestWithLatestChildrenAndPropagateVersions
Scoped Sets are currently not supported with this repository update type
api for DDI XML fragment and DDI FragmentInstance sets
In Elastic, Index abstract, purpose, contributors, creators, publishers, subjects, keywords, related organizations (as extraText)
For CVs, index plain values and multilingual matched category information in separate fields
Enable file uploads
Calculate SHA256 and MD5 of deposited files
Changed¶
use problem details and proper http status codes for known repository errors
use one combined query for repository statistics
In postgresql search, remove secondary sorting by alpha for non alphabetical sorts
don’t calculate metadata rank unless RankResults is true or sorting by metadata rank
Increase timeout of database transaction scopes
Ensure StudyAbstract has text indexed
count outdated parents as user submitted, so a count of outdated can be determined before version propagation
don’t include duplicates in update repository to latest children
add text extract for individuals
use a DB transaction for sqlite in registeritems instead of a general transactionscope
explicitly set all latest fields in sql server upgrade script
Fixed¶
Prefix search on postgresql
only allow one prefix in a search term on postgresql
always set format to RepositoryFormats.Ddi33 when rewriting repository items in transactions
Index actual values of subjects and keywords
handle non concrete items in transaction propagation
cleanup text extracts, no duplicate names and labels
script to reset sql server latest flag
Colectica SDK¶
Added¶
nuget targets for both net472 and netstandard2
Map more content from DDI 2 to DDI 3
DDI 2 reading: support multiple fileDscr elements, creating multiple PhysicalInstances
nuget package for Colectica.Reporting
Upgraded HtmlAgilityPack
TimeoutInSeconds property, used for REST clients
For CategoryStatistics, allow both Frequency and WeightedFrequency
For DDI 2, import more summary statistics, including weighted frequencies
Give a label to Instruments that are created during DDI 2 Import
Add GetRepositoryStatistics to RepositoryClientBase and its implementations
Add methods to detect DDI format from XML content or streams, in addition to files
item indexer pool and interface
DDI Item indexer addin support
flag to switch off change tracking with static VersionableBase.DisableChangeTracking
add a SucceededBy property to Iversionable
Make the selected remote repository available to addins
add VariableGroups to PhysicalInstance
Changed¶
In DDI 3.3, do not store OtherMaterial CustomFields on the parent item
DDI 2 reader: for varGrps with no variables, don’t try to locate a PhysicalInstance based on the variables
DDI 2 mapping updates
DDI 2 look for a fileDscr ID in more places.
Remove Repository dependency from ViewModel assembly
Move LocalCheckout to Model assembly to enable that
Use RepositoryClientBase in ViewModel project. Pass in LocalRepositoryClient from callers as needed.
In DDI 2 import, support date ranges with ‘YYYY’ for Year, instead of full dates
Safe handling for URI creation in DDI 2 import
Treat two multilingual strings with HasValue == false as equal
DDI 2 import multiple OtherMaterials when they exist
DDI 2 import Option to always create a wrapper sequence for questions that have pre/post text
When ingesting DDI 2, use the designated language, not .Current
3.1 Questacy import, gracefully handle incorrectly created identifiers and references
Fixed¶
Do not add DataCollection to StudyUnit twice in DDI 2 import
DDI 2 import: For year-only dates, set the DDI 3 Date’s precision correctly
add UnitType to GetChildren, RemoveChild, ReplaceChild in RepresentedVariable
Colectica Questionnaires and Blaise Colectica Questionnaires¶
Added¶
History view and version comparison in Questionnaires
In Open window, show instruments in alphabetical order
When looking for version state, only use the first configured repository, not all. Consider this the “active” repository
Show Instructions in preview cards
Edit name and label for display logic
For numbers, use validated text boxes, instead of the numeric-up-down spinner controls
Allow adding multiple existing items at once
Load IVersionableCommands that work against Instruments
checks when adding existing items
Colectica for Excel¶
Added¶
handle datetime variables from SAS in variable statistics computations
Changed¶
skip tagged missing values in SAS code list when the list has duplicate entries for one value
Fixed¶
Microsoft re-signed the old vc9 installer, new certificate added in wix remote payloads
Microsoft re-signed the net 472 installer, new certificate in wix remote payloads
Colectica Report Generation¶
Added¶
Translations for built in reporting strings
Changed¶
Show up to two decimal points in percentages
In PDFs, show “0” as a frequency, instead of blank
For PDF output for variable statistics, for numbers that have more than a few zeros after the decimal separator, show in scientific notation
Fixed¶
LogicalProduct reports: include all child items
Don’t add code lists twice under a variable that does not have summary statistics
Colectica Workflow¶
Added¶
New workflow website based on netcore
New workflow service task runner based on netcore
workflow replication of files from source to target repository
enable forward headers
During workflow approval process, let user choose which files to replicate
Show item identifiers as tooltips in the process list
Configurable command timeout
Show all languages of item labels/names in Workflow Web
Changed¶
separate Portal database from portal website project
load replication repositories by configuration
Allow repository to take a database connection
Allow workflow replication db to take a database connection
create task runner to process workflow replication events
new workflow website api to handle replication requests from repositories
re-written DataAccess, use passed in connection strings and providers, remove oracle
new database strings connection class
ToIdentifier for identifierinrequest
Remove old workflow web
add estonian to old cache builder
calculate preview db path better
Fixed¶
only VACUUM when isNpgsql