Shared Table Profile
- Kevin Wu
This section describes the Shared Table profile. This profile enables workflow instances to share tables for lookups.
Shared Table
Using the Table Lookup Service instead of adding tableCreate in each workflow instance will increase the throughput with fewer duplicated tables, fewer lookups, and reduced memory consumption.
The Table Lookup Service comprises a profile in which SQL queries are defined, and two APL functions; one that references the profile and creates a shared table, and one that can be used for refreshing the table data from the APL code.
The Shared Table profile is loaded when you start a workflow that depends on it. Changes to the profile become effective when you restart the workflow each time you save the profile.
Memory Allocation
There are three different ways to allocate memory for the created tables. By default, the tables are kept as Java objects in memory. The shared tables can also be configured to keep the tables as raw data either on or off the heap. By using raw data, the overhead of Java objects is removed and less memory is required.
The type of memory allocation chosen for the shared tables is configured in the Shared Table profile by selecting a Table Storage parameter. If relevant, you can select an Index Storage parameter and also Variable Width Varchar Columns. For further information about these settings, see the section below, Shared Table Profile Configuration.
For further information about memory allocation, see the System Administration Guide.
Shared Table Profile Configuration
The Shared Table profile configuration is opened by clicking on the New Configuration button in Desktop and selecting the Shared Table Profile option.
The Shared Table profile dialog
The contents of the buttons in the button bar may change depending on which configuration type has been opened. The Shared Table profile uses the standard menu items and buttons that are visible for all configurations, and these are described in Build View.
The Shared Table profile dialog contains the following configurations:
Setting | Description |
|---|---|
Database | |
Database | Click the Browse...button and select the Database profile you want to use. Any type of database that has been configured in a database profile can be used. See Database Profile for further information. |
Release Timeout (seconds) | If this check box is selected, the table will be released when the entered number of seconds has passed since the workflows accessing the table were stopped. The entered number of seconds must be larger than 0. If this check box is not selected, the table will stay available until the execution context is restarted. |
Refresh Interval (seconds) | Select this check box in order to refresh the data in the table with the interval entered. The entered number of seconds must be larger than 0. If this check box is not selected, the table will only be refreshed if the APL function Note!The interval for checking if the table needs to be refreshed is 10 seconds, which is the minimum time before a new refresh is performed. If a refresh fails, an error is generated in the system log, but the table is not cleared - the old data remains in the shared table. A new refresh is initiated every 10th second until the refresh has finished successfully. |
Table Storage | |
Object | Select this option to set the Table Storage to Object. If you select this option, the shared tables are stored as Java objects on the JVM heap. Note!If you have selected to use a profile with the CSV database type, this is the only option available if you have not configured properties using Advanced Connection Setup. |
On Heap | Select this option to set the Table Storage to On Heap. If you select this option, the shared tables are stored in a compact format on the JVM heap. If you select On Heap, you must select an option for the Index Storage. Note!If you have selected to use a profile with the CSV database type, this option is only available if you have configured properties for this using Advanced Connection Setup. |
Off Heap | Select this option to set the Table Storage to Off Heap. If you select this option, the shared tables are stored in a compact format outside the JVM heap. If you select Off Heap, you must select an option for the Index Storage. |
Unsafe | Select this option to set the Table Storage to Unsafe. If you select this option, the shared tables are stored in a compact format. If you select Unsafe, you must select an option for the Index Storage. |
Primitive Lookup | Select this option to set the Table Storage to Primitive Lookup. This provides simple lookup tables with a fast lookup function but they are limited to two columns of type |
Index Storage | |
Object | Select this option to set the Index Storage to Object. If you select this option, the index is stored as Java objects on the JVM heap. This option is only available if you have selected On Heap, Off Heap or Unsafe for Table Storage. |
Pointer | Select this option to set the Index Storage to Pointer. If you select this option, the index is stored as pointers to the table data. This option is only available if you have selected On Heap, Off Heap or Unsafe for Table Storage. |
Cached Long/Int Pointer | Select this option to set the Index Storage to Cached Long/Int Pointer. This option is only available if you have selected On Heap, Off Heap or Unsafe for Table Storage. For numeric index columns, the Cached Long/Int Pointer can be used for faster lookups, but at the cost of slightly higher memory consumption. |
Variable Width Varchar Columns | Select this check box to enable variable width storage of varchar columns. This reduces memory usage for columns that are wide and of varying widths. |
SQL Load Statement | |
SQL Load Statement | In this field, an SQL SELECT statement should be entered in order to create the contents of the table returned by the The following statement will return a table named MyTable with the columns key and value when the If no data has been fetched from the database, SQL errors in the table lookup will cause runtime errors (workflow aborts). However, if data has already been fetched from the database then this data will be used. This will also be logged in the System Log. Whenever possible, use values of the type long instead of the type date or string. This may improve performance. |
Table Indices | If you want to create an index for one or several columns of the shared table, these columns can be added in this field by clicking the Add button and add the columns for which you want to create an index. The index will start with 0 for the first column. Create indices in the Shared Table profile based on the data that is fetched from the database. Even if you look up multiple columns in APL, using one index instead of several may result in improved performance. |
APL
The following functions are included in the Table Lookup Service:
tableCreateSharedtableRefreshShared
tableCreateShared
This function returns a shared table that holds the result of the database query entered in the Shared Table profile.
table tableCreateShared
( string profileName
boolean disableCommit )
Parameters:
Returned Value | Description |
|---|---|
| Name of the Shared Table profile you want to use. |
disableCommit | An optional parameter to disable the commit statement from being performed at the end of every SQL transaction for this particular function. Setting this parameter to false will result in the commit statement being performed at the end of every SQL transaction for this particular function. By default, disableCommit is set to true unless otherwise changed via this parameter. |
Returns | A table containing the columns stated with the SQL query in the stated Shared Table profile, that can be shared by several workflow instances. |
tableRefreshShared
This function can be used for refreshing the data for a shared table configured with a Shared Table profile. The table will be updated for all workflow instances that are using the table and running on the same EC.
table tableRefreshShared
( string profileName
boolean disableCommit )
Parameters:
Returned Value | Description |
|---|---|
| Name of the Shared Table profile you want to refresh data for. |
disableCommit | An optional parameter to disable the commit statement from being performed at the end of every SQL transaction for this particular function. Setting this parameter to false will result in the commit statement being performed at the end of every SQL transaction for this particular function. By default, disableCommit is set to true unless otherwise changed via this parameter. |
Returns | A refreshed shared table. |
Additional Performance Tuning
The Oracle JDBC driver includes a feature built that allows you to set the number of rows that are prefetched while the full result set is being formulated. At the time of writing, the default number of prefetched rows is 10. You can increase this value by setting the Execution Context property oracle.jdbc.defaultRowPrefetch in the relevant Pico configuration file.