9.14.6 Data Veracity Performance Tuning

Tuning on Oracle 12c

This section describes how to tune the Oracle Database for use with Data Veracity. 

Data Veracity uses Oracle 12c's JSON capabilities for storing data. This provides the capability to improve the performance of the SQL queries from Data Veracity, by applying the Oracle's database features for JSON fields indexing. The following are suggestions to how you may improve performance for Data Veracity, it is highly recommended that you consult your DBA before proceeding with any of these suggestions or if they are necessary in your solution.

Example of a table created in Data Veracity, viewed using sqlplus

The Table above shows a table, created using the SQL generation in the Data Veracity profile. The UDR_JSON field is where the record for every UDR is kept. The UDR is kept in a JSON format and Oracle 12c has a way to allow users to create an index of a field located in the JSON format itself.

Example UDR with 3 sub UDR levels

Take for example, when the user frequently searches for the UDR field intField1, the search performance could be improved by creating an index in the Oracle database for this field. The following will be an SQL command to create a unique index meant for a field in the JSON record.

CREATE UNIQUE INDEX dvIntField1_idx
ON dvtestudr (json_value(udr_json, 
'$.intField1' 
RETURNING NUMBER ERROR ON ERROR));

If fields located within multiple sublevels of UDR are frequently queried by the user, an index could be created to improve the performance for searching this field. The following SQL command will create a unique index for a field that is located in within the subUDRs. 

CREATE UNIQUE INDEX dvIntSubField2_idx
ON dvtestudr (json_value(udr_json, 
'$.sub1Field.sub2Field.intField' 
RETURNING NUMBER ERROR ON ERROR));

Tuning on PostgreSQL 9.4

This section describes how to tune the PostgreSQL Database for use with Data Veracity.

Much like Oracle, Data Veracity allows the user to improve the performance of PostgreSQL queries. The following will be a suggestion on how to improve the performance. It is always important to consult your DBA before proceeding with any of these suggestions or if they are necessary in your solution.

PostgreSQL has certain specific syntaxes that are used for certain data types, the following examples will show how to index in PostgreSQL for string data types, integer data types and timestamps.

Following from the example as shown above, This command will show how a string field can be indexed.

CREATE INDEX dvStrField1 ON dvtestudr ((udr_json ->> 'stringField1'));

For fields located within multiple sublevels of UDR, the following command will create a unique index for a field that is located in within the subUDRs.

CREATE INDEX dvStrField1 ON dvtestudr ((udr_json -> 'sub1Field' -> 'sub2Field' ->> 'stringField1'));

Following from the example as shown above, to index the UDR field intField 2, the search performance could be improved by creating an index in the PostgreSQL database for this field. The following will be an SQL command to create a unique index meant for an integer field in the JSON record.

CREATE INDEX dvIntField1 ON dvtestudr (cast(udr_json ->> 'intField1' AS int));

When it comes to timestamps, a function is the recommended suggestion to make the data type immutable when it is casted for an index. The example will show a sample PostgreSQL function and the command for the index.

CREATE OR REPLACE FUNCTION dvcasttime(text)
RETURNS timestamptime AS
$$SELECT to_timestamp($1, 'YYYY-MM-DD HH24:MI:SS')$$ -- change the timestamp format to the format being used in the field
LANGUAGE sql IMMUTABLE;

CREATE INDEX dvdDateField1 ON dvtestudr (dvcasttime(udr_json ->> 'dateField1'));