2.13 Enabling Scalable Batch Transaction Service

The default behavior of  is to store batch transaction state information in a database, i e Derby,Oracle, or PostgreSQL.

The Platform creates transaction ids for each batch as a reference that workflows can use when requesting state changes.

If the handling of transaction state becomes a bottleneck, e g in case of shared filesystem, it is preferable to persist the information on the EC filesystem rather than the database. A centralized function is still needed for generation of transaction ids, since it is important that these are unique and in ascending order. This can be achieved by enabling Scalable Batch Transaction Service (SBTS).

When SBTS is enabled, the generation of transactions is handled by the standard service txn-id-service and you can select the transaction data storage for each workflow configuration:

• Database
• Local EC (file)

The settings above are available in the Desktop. For further information, see 3.1.8.5 Execution Tab in the Desktop User's Guide.

The txn-id-service may run on one or up to three SCs.

When you are running the txn-id-service service on multiple SCs, only one is responsible for generating transaction ids. This service is Akka-based and it is recommended to monitor and remove unresponsive SC from the cluster to avoid "split-brain" scenarios, i e two or more nodes generating inconsistent transaction ids. The name of the Akka cluster is txn-id-cluster. For further information, see 2.7.6 Managing Akka Cluster Failure.

To enable SBTS:

1. The service runs on SCs that have the value platform in the property mz.servicehost.natures, e g the pre-configured SC psc1. Run the following commands if you want to create another SC that hosts the service:

   $ mzsh topo set topo://container:<container>/pico:<sc name> 'template:mz.standard-sc'
   $ mzsh topo set topo://container:<container>/pico:<sc name>/obj:config.properties '{
    mz.servicehost.natures="platform"
    mz.servicehost.port.range="<start port>-<end port>"
   }'

     

2. The platform will be restarted in one the following steps. For this reason, it is recommended that you disconnect all ECSAs in order to keep real-time workflows running.

   $ mzsh mzadmin/<password> disconnect <ecsa name> ...

    

3. Disable all workflow groups from the Execution Manager in Desktop, or by entering an mzsh command, for example:

   $ mzsh mzadmin/<password> wfgroupdisable \*

   Note!

   If you use the wfgroupdisable command, make sure that you enable all system tasks again when you are done.

4. Stop all batch workflows and let them finish execution.
    

5. Shut down all connected Desktops.

   If you want to see which Desktops that are connected, you can use the following command:

   $ mzsh mzadmin/<password> pico -view

     

6. Shut down all ECs the ECSAs that have not been disconnected.

   $ mzsh shutdown <ec name> <ecsa name>

7. Make sure that the Platform is running.

   $ mzsh status platform

8. Start the SC(s) that will run the service, e g psc1.

   $ mzsh startup <sc name>

     

9. Run the script enable-sbts.sh in MZ_HOME/scripts/system-configure from the Platform Container. The script adds the standard service txn-id-service to the STR. Once the script is completed, generation of transaction ids will be handled by the SC instead of the Platform. 

   $ cd $MZ_HOME/scripts/system-configure
   $ ./enable-sbts.sh mzadmin <password>

     

10. Restart the platform and the SCs that will run the  SBTS service.

    $ mzsh shutdown platform <sc names>
    $ mzsh startup platform  <sc names>

11. Start the standard services.

    $ mzsh service start --scope standard

     

12. Start all EC/ECSAs that were previously shut down.

    $ mzsh startup <ec names> <ecsa names>

13. Enable the workflow groups.
      

14. You can now configure where the transaction state information should be stored for each workflow configuration, using the Workflow Properties in Desktop. For further information, see 3.1.8.5 Execution Tab in the Desktop User's Guide.