Distributed Storage Functions(3.3)

The Distributed Storage Profile enables you to access a distributed storage solution from APL without having to provide details about its type or implementation.

The APL functions described in this section use a key-value model to read, store and remove data via the specified profile. The functions can optionally use sessions that are either committed or rolled back in order to secure transaction consistency.

Initializing Storage

dsInitStorage

The dsInitStorage function returns a Distributed Storage Profile object that is used in subsequent APL calls to distributed storage functions.

any dsInitStorage(string profid)
ParameterDescription

profid

A string containing a reference to a Distributed Storage Profile

Returns

A Distributed Storage Profile object

Example - Using dsInitStorage

any storage=null;

initialize {
    storage = dsInitStorage("default.ds_profile"); 

    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

Note!

The dsInitStorage function must be used in the initialize block and not consume block.

Storing and Removing Data

The following functions  enable you to perform changes on data in a distributed storage:

  • dsStore
  • dsStoreUdr
  • dsStoreMany
  • dsStoreManyUdrs
  • dsRemove
  • dsRemoveMany

dsStore

The dsStore function stores a key-value pair in a distributed storage.

void dsStore(
 any profid, 
 string key, 
 bytearray value, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

key

The key of a key-value pair

value

The value of a key-value pair

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to store data in non-transaction mode. If the identifier is set the transaction is completed by calling dsCommitTransaction.

Returns

Nothing

Example - Using dsStore

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume { 
    bytearray myValue = null;
    strToBA(myValue, "123");
    dsStore(storage, "mykey", myValue, null); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    } 
}

dsStoreUdr

The dsStoreUdr function stores a key-value pair in a distributed storage.

void dsStoreUdr(
 any profid, 
 string key, 
 drudr myUDR, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

key

The key of a key-value pair

myUDR

The UDR to be stored

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to store data in non-transaction mode. If the identifier is set the transaction is completed by calling dsCommitTransaction.

Returns

Nothing

Example - Using dsStoreUDR

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume { 
    default.dataUDR udr = udrCreate(default.myultra.dataUDR);    
    dsStoreUdr(storage, "mykey", udr, null); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
 }
}

dsStoreMany

The dsStoreMany function stores a set of key-value pairs in a distributed storage.

void dsStoreMany(
 any profid, 
 map<string, bytearray> values, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

values

The key-value pairs to store

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to store data in non-transaction mode. If the identifier is set the transaction is completed by calling dsCommitTransaction.

Returns

Nothing

Example - Using dsStoreMany

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }     
}
 
consume {
    map<string, bytearray> myValues = 
        mapCreate(string, bytearray);
    bytearray value1; 
    bytearray value2;
    strToBA(value1, "abc");
    mapSet(myValues,"first", value1);
    strToBA(value2, "def");    
    mapSet(myValues,"second", value2); 
    dsStoreMany(storage, myValues, null);
    if(null != dsLastErrorMessage()) {
        //Error Handling
    } 
}

dsStoreManyUdrs

The dsStoreManyUdrs function stores a set of key-value pairs in a distributed storage.

void dsStoreManyUdrs(
 any profid, 
 map<string, drudr> myUDRs, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

myUDRs

The key-value pairs to store

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to store data in non-transaction mode. If the identifier is set the transaction is completed by calling dsCommitTransaction.

Returns

Nothing


Example - Using dsStoreManyUDRs

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");
    if(null != dsLastErrorMessage()) {
        //Error Handling
    } 
}
 
consume { 
    map<string, drudr> myUDRs = mapCreate(string, drudr);
    default.dataUDR udr1 = udrCreate(default.myultra.dataUDR); 
    mapSet(myUDRs,"first", udr1);
    default.dataUDR udr2 = udrCreate(default.myultra.dataUDR); 
    mapSet(myUDRs,"second", udr2); 
    dsStoreManyUdrs(storage, myUDRs, null); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    } 
}

dsCommand

Note!

This function is only valid for Couchbase.

The dsCommand function stores data with a Time To Live (TTL) on the entry by updating the ‘expiration’ field for data in Couchbase, meaning it will be removed when it gets too old.

void dsCommand(
 any profid, 
 string "ttlStore"
 string key, 
 int ttl,
 drudr myUDR)

You can also use the dsCommand to extend the expiration field of the data in Couchbase using the “touch” action.

void dsCommand(
 any profid, 
 string "touch"
 string key, 
 int ttl)

Parameter

Description

profid

The Distributed Storage profile.

action

The action you want to perform, “ttlStore” for storing UDRs with TTL or “touch“ to extend the TTL.

key

The key of a key-value pair.

ttl

Time To Live in seconds.

myUDR

The UDR to be stored.

Returns

Nothing.

Example - Using dsCommand

int ttl = 50;
any result = dsCommand(storage, "ttlStore", "3333", ttl,input);
debug(result);
sleep(10000);
result = dsCommand(storage, "touch", "3333", ttl);

dsRemove

The dsRemove function removes a key-value pair, identified by a key, from a distributed storage.

void dsRemove(
 any profid, 
 string key,   
 any transid)
ParameterDescription

profid

The Distributed Storage profile

key

The key of the key-value pair to remove

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to remove data in non-transaction mode. If the identifier is set the transaction is completed by calling dsCommitTransaction.

Returns

Nothing

Example - Using dsRemove

any storage = null;
initialize {
    storage = dsInitStorage("default.ds_profile");
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }     
}
 
consume {
    //It is assumed here that "mykey" is already stored.
    dsRemove(storage, "mykey", null);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

dsRemoveMany

The dsRemoveMany function removes a list of key-value pairs, identified by a list of keys, from a distributed storage.

void dsRemoveMany(
 any profid, 
 list<string> keys,
 any transid)
ParameterDescription

profid

The Distributed Storage profile

keys

The keys of the key-value pairs to remove

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to remove data in non-transaction mode. If the identifier is set the transaction is completed by calling dsCommitTransaction.

Returns

Nothing

Example - Using dsRemoveMany

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }     
}
 
consume {
    list<string> myKeys = listCreate(string);
    int i = 0;
    // It is assumed here that "KEY0", "KEY1".."KEY9" 
    // are already stored. 
    while (i < 10) {
        listAdd(myKeys, "KEY" + (i));
        i = i + 1;
    }
    dsRemoveMany(storage, myKeys, null);
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

Reading Data

The following functions  enable you to read data from a distributed storage.

  • dsRead
  • dsReadUdr
  • dsReadMany
  • dsReadManyUdrs

dsRead

The dsRead function reads a value, identified by a key, from a distributed storage.

bytearray dsRead(
 any profid, 
 string key, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

key

The key of a key-value pair

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to read data in non-transaction mode. If the identifier is set, the read key-value pair is locked for other transactions untildsCommitTransaction or dsRollbackTransaction is called.

Returns

A bytearray containing the value of a key-value pair

Example - Using dsRead

any storage = null;
initialize {
    storage = dsInitStorage("default.ds_profile"); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
consume { 
    //It is assumed here that "mykey" is already stored.
    bytearray myValue = dsRead(storage, "mykey", null);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

dsReadUdr

The dsReadUdr function reads a set of UDRs, identified by a list of keys, from a distributed storage.

drudr dsReadUdr(
 any profid, 
 string key, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

key

The key of a key-value pair

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to read data in non-transaction mode. If the identifier is set, the read key-value pair is locked for other transactions untildsCommitTransaction or dsRollbackTransaction is called.

Returns

A UDR identified by the key parameter

Example - Using dsReadUDR

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile"); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume {
    //It is assumed here that "mykey" is already stored.
    default.myultra.dataUDR udr = 
        (default.myultra.dataUDR) dsReadUdr(storage, "mykey", 
        null); 
    if(null != dsLastErrorMessage()) {
       //Error Handling
    }
}

dsReadMany

The dsReadMany function reads a set of key-value pairs, identified by a list of keys, from a distributed storage.

map<string, bytearray> dsReadMany(
 any profid, 
 list<string> keys,
 any transid)
ParameterDescription

profid

The Distributed Storage profile

keys

The keys of the key-value pairs to be read

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to read data in non-transaction mode. If the identifier is set, the read key-value pairs are locked for other transactions untildsCommitTransaction or dsRollbackTransaction is called.

Returns

A map containing the key-value pairs

Example - Using dsReadMany

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile"); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume {
    list<string> myKeys = listCreate(string);
    int i = 0;
    // It is assumed here that "KEY0", "KEY1".."KEY9" 
    // are already stored.
    while (i < 10) {
        listAdd(myKeys, "KEY" + (i));
        i = i + 1;
    }
    map<string, bytearray> myMap = mapCreate(string, bytearray);
    myMap = dsReadMany(storage, myKeys, null); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

dsReadManyUdrs

The dsReadManyUdrs function reads a key-value map, identified by a key, from a distributed storage.

map<string, drudr> dsReadManyUdrs(
 any profid, 
 list< string keys, 
 any transid)
ParameterDescription

profid

The Distributed Storage profile

keys

The keys of the key-value pairs to be read

transid

The transaction identifier that is returned by dsBeginTransaction. Use a null value to read data in non-transaction mode. If the identifier is set, the read key-value pairs are locked for other transactions untildsCommitTransaction or dsRollbackTransaction is called.

Returns

A map containing the key-value pairs

Example - Using dsReadManyUDRS

any storage = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile"); 
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume {
    list<string> myKeys = listCreate(string);
    int i = 0;
    // It is assumed here that "KEY0", "KEY1".."KEY9" 
    // are already stored.
 
    while (i < 10) {
        listAdd(myKeys, "KEY" + (i));
        i = i + 1;
    }
    map<string, default.myultra.dataUDR> myUdrMap =  
        mapCreate(string, default.myultra.dataUDR);
    myUdrMap = dsReadManyUdrs(storage, myKeys, null);   
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

Transaction Functions

The following functions provide transaction safety by performing commit or rollback on a set of calls to a Distributed Storage Profile:

  • dsBeginTransaction
  • dsCommitTransaction
  • dsRollbackTransaction

dsBeginTransaction

The dsBeginTransaction function begins a new transaction and returns an object that can be used in subsequent APL calls to distributed storage functions.

any dsBeginTransaction(any profid)
ParameterDescription

profid

The Distributed Storage profile

Returns

A transaction identifier

dsCommitTransaction

The dsCommitTransaction function ends a transaction and commits changes that has been made using the specified transaction identifier.

void dsCommitTransaction(
 any profid,
 any transid)
ParameterDescription

profid

The Distributed Storage profile

transid

The transaction identifier that is returned by dsBeginTransaction

Returns

Nothing

Example - Using dsCommitTransaction

any storage = null;
any transid = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume {
    transid = dsBeginTransaction(storage);
    bytearray myValue = null;
    strToBA(myValue, "123");
    dsStore(storage, "mykey1", myValue, transid);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
    dsStore(storage, "mykey2", myValue, transid);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
    dsCommitTransaction(storage, transid);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

dsRollbackTransaction

The dsRollbackTransaction function ends a transaction and reverts changes that has been made using the specified transaction identifier.

void dsRollbackTransaction(
 any profid,
 any transid)
ParameterDescription

profid

The Distributed Storage profile

transid

The transaction identifier that is returned by dsBeginTransaction

Returns

Nothing

Example - Using dsRollbackTransaction

any storage = null;
any transid = null;
 
initialize {
    storage = dsInitStorage("default.ds_profile");    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
 
consume {
    transid = dsBeginTransaction(storage);
    bytearray myValue = null;
    strToBA(myValue, "123");
    dsStore(storage, "mykey1", myValue, transid);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
    dsStore(storage, "mykey2", myValue, transid);    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    } 
    dsRollbackTransaction(storage, transid); //nothing is stored    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}

Iterator Functions

The following functions  are used for traversing through a set of keys in a distributed storage:

  • dsCreateKeyIterator
  • dsCreateREKeyIterator
  • dsGetNextKey
  • dsDestroyKeyIterator

dsCreateREKeyIterator

Note!

This iterator function is only valid for Redis.

The dsCreateREKeyIterator function creates an iterator object that is used in subsequent APL calls to other iterator functions.

any dsCreateREKeyIterator(
 any profid,
 string searchpattern)
ParameterDescription

profid

The Distributed Storage profile

searchpattern

A search pattern which may include wild card '*'

Returns

An iterator object used in subsequent APL calls to dsGetNextKey and dsDestroyKeyIterator.

dsGetNextKey

The dsGetNextKey function gets the next available key from a distributed storage using an iterator object. Each subsequent call to the function returns the next key in the iterator's range.

string dsGetNextKey(
 any profid,
 any iterator)
ParameterDescription

profid

The Distributed Storage Profile.

iterator

The iterator object returned by dsCreateKeyIterator.

Returns

The key of a key-value pair. A null value is returned if a key cannot be found.

Example - Using dsCreateKeyIterator

any storage = null;
initialize {
    storage = dsInitStorage("default.ds_profile");    
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}
consume { 
    string startKey = "mystartkey";
    string stopKey = "mystopkey";     
    any iterator = 
        dsCreateKeyIterator(storage, startKey, stopKey);
    string key = dsGetNextKey(storage, iterator);       
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }     
    // It is assumed here that "mystartkey" and "mystopkey" 
    // are already stored
    while ( null != key) {
        debug(dsRead(storage, key, null));
        if(null != dsLastErrorMessage()) {
            //Error Handling
        }
        key = dsGetNextKey(storage, iterator);
        if(null != dsLastErrorMessage()) {
            //Error Handling
        }
    }
    dsDestroyKeyIterator(storage, iterator);
    if(null != dsLastErrorMessage()) {
        //Error Handling
    }
}


dsDestroyKeyIterator

The dsDestroyKeyIterator function destroys an iterator object created with dsCreateKeyIterator in order to free up memory. It is good practice to always include a call to this function in the APL code though it may not be required for all types of storage.

void dsDestroyKeyIterator( 
 any profid,
 any iterator)
ParameterDescription

profid

The Distributed Storage profile

iterator

The iterator object returned by dsCreateKeyIterator

Returns

Nothing

Error Handling

dsLastErrorMessage

The dsLastErrorMessage function returns the error message from the last call to a distributed storage function. Distributed storage functions that are not used for error handling, do not return error codes and generally do not cause runtime errors. For this reason, The dsLastErrorMessage function must be called after each operation to validate success.

string dsLastErrorMessage()
ParameterDescription

Returns

An error message, or null if no error has occurred since the last call to this function. The content of message depends on the type of profile that is used with the Distributed Storage profile.