Elasticsearch Indices

Colectica Portal uses Elasticsearch for two optional features: full text, faceted search; and data extraction. Both these features are optional.

This document describes the Elastic search indices, documents, and properties used by Colectica Portal. For information on populating Elasticsearch, see Elastic Indexer.

Elasticsearch for Data Extraction

Note

Colectica may change the storage mechanism used to power its custom data extract functionality in the future.

See also

For details on Elasticsearch column store capabilities, see https://www.elastic.co/blog/elasticsearch-as-a-column-store

Elastic Index

An Elasticsearch index is created for each PhysicalInstance that has its corresponding data file ingested for the custom data extract functionality. The name of the index is {prefix}_physicalinstance_{identifier}.

  • {prefix} is the value specified in the Elasticsearch:IndexName value of the appsettings.json configuration file. See /portal/technical/deployment/configuration for details.

  • {identifier} is the identifier of the PhysicalInstance.

Documents and Identification of Data

Each row of the data file is indexed as an Elasticsearch document. The identifier of the document is the value of the in the columns corresponding to the variable(s) specified as the CaseIdentifier.

Properties for Data

Each variable in the PhysicalInstance is mapped as a property, with its type specified as either text or numeric. The content of each property is the datum at the appropriate row and column of the dataset that correspond to the Elasticsearch document and property, respectively.

Running on alternative versions of Elasticsearch

Colectica currently supports Elasticsearch version 7.x. Historically elasticsearch has not maintained compatible across their major versions.

Starting with Elasticsearch 8.x, the Elasticsearch server provides some backwards compatibility using a version header in its REST api calls.

To try to use a newer version of Elasticsearch with Colectica, you can enable this version header by setting the ElasticIndexer:EnableApiVersioningHeader setting to true in the ElasticIndexer and the Elasticsearch:EnableApiVersioningHeader setting to true in the Colectica Repository.