Create model execute command (#680)

* Create model execute command (in addition to the model run command, which is deprecated for enterprise cluster)

* Refactor - put AI.MODELRUN functions in a separate file called deprecated.c

* Small fix for valgrind

* PR fixes.
Call the function that validates that a key is located in the current shard from within RAI_GetTensoe/ModelFromKeyspace (relevant to enterprise only).

* Documentation

* More PR fixes

* - Add modelstore command (new version for modelset) and move modelset to depracted.c

* Refactor tests: use the new commands instead of the deprecated ones. Use "tests_commands" to test the new commands syntax (not for a specific backend), and test the deprecated commands only in "test_deprecated".
Update AI.MODELSTORE documentation.

* PR fixes + fix leak in onnx modelrun

* Fix in LLAPI test module loading

* fix format + leak in onnx

Co-authored-by: DvirDukhan <[email protected]>

Author: alonre24

README.md

@@ -46,7 +46,7 @@ Note that Redis config is located at `/usr/local/etc/redis/redis.conf` which can
 
 On the client, set the model
 ```sh
-redis-cli -x AI.MODELSET foo TF CPU INPUTS a b OUTPUTS c BLOB < tests/test_data/graph.pb
+redis-cli -x AI.MODELSTORE foo TF CPU INPUTS 2 a b OUTPUTS 1 c BLOB < tests/test_data/graph.pb
 ```
 
 Then create the input tensors, run the computation graph and get the output tensor (see `load_model.sh`). Note the signatures:

docs/commands.md

@@ -152,15 +152,15 @@ redis> AI.TENSORGET mytensor META BLOB
 !!! important "Using `BLOB` is preferable to `VALUES`"
     While it is possible to get the tensor as binary data or numerical values, it is recommended that you use the `BLOB` option. It requires fewer resources and performs better compared to returning the values discretely.
 
-## AI.MODELSET
-The **`AI.MODELSET`** commands stores a model as the value of a key.
+## AI.MODELSTORE
+The **`AI.MODELSTORE`** command stores a model as the value of a key.
 
 **Redis API**
 
 ```
-AI.MODELSET <key> <backend> <device>
-    [TAG tag] [BATCHSIZE n [MINBATCHSIZE m] [MINBATCHTIMEOUT t]]
-    [INPUTS <name> ...] [OUTPUTS name ...] BLOB <model>
+AI.MODELSTORE <key> <backend> <device>
+    [TAG tag] [BATCHSIZE n [MINBATCHSIZE m [MINBATCHTIMEOUT t]]]
+    [INPUTS <input_count> <name> ...] [OUTPUTS <output_count> <name> ...] BLOB <model>
 ```
 
 _Arguments_
@@ -176,11 +176,13 @@ _Arguments_
     * **GPU**: a GPU device
     * **GPU:0**, ..., **GPU:n**: a specific GPU device on a multi-GPU system
 * **TAG**: an optional string for tagging the model such as a version number or any arbitrary identifier
-* **BATCHSIZE**: when provided with an `n` that is greater than 0, the engine will batch incoming requests from multiple clients that use the model with input tensors of the same shape. When `AI.MODELRUN` is called the requests queue is visited and input tensors from compatible requests are concatenated along the 0th (batch) dimension until `n` is exceeded. The model is then run for the entire batch and the results are unpacked back to the individual requests unblocking their respective clients. If the batch size of the inputs to of first request in the queue exceeds `BATCHSIZE`, the request is served immediately (default value: 0).
-* **MINBATCHSIZE**: when provided with an `m` that is greater than 0, the engine will postpone calls to `AI.MODELRUN` until the batch's size had reached `m`. In this case, note that requests for which `m` is not reached will hang indefinitely (default value: 0), unless `MINBATCHTIMEOUT` is provided.
-* **MINBATCHTIMEOUT**: when provided with a `t` (expressed in milliseconds) that is greater than 0, the engine will trigger a run even though `MINBATCHSIZE` has not been reached after `t` milliseconds from the time a `MODELRUN` (or the enclosing `DAGRUN`) is enqueued. This only applies to cases where both `BATCHSIZE` and `MINBATCHSIZE` are greater than 0.
-* **INPUTS**: one or more names of the model's input nodes (applicable only for TensorFlow models)
-* **OUTPUTS**: one or more names of the model's output nodes (applicable only for TensorFlow models)
+* **BATCHSIZE**: when provided with an `n` that is greater than 0, the engine will batch incoming requests from multiple clients that use the model with input tensors of the same shape. When `AI.MODELEXECUTE` (or `AI.MODELRUN`) is called the requests queue is visited and input tensors from compatible requests are concatenated along the 0th (batch) dimension until `n` is exceeded. The model is then run for the entire batch and the results are unpacked back to the individual requests unblocking their respective clients. If the batch size of the inputs to of first request in the queue exceeds `BATCHSIZE`, the request is served immediately (default value: 0).
+* **MINBATCHSIZE**: when provided with an `m` that is greater than 0, the engine will postpone calls to `AI.MODELEXECUTE` until the batch's size had reached `m`. In this case, note that requests for which `m` is not reached will hang indefinitely (default value: 0), unless `MINBATCHTIMEOUT` is provided.
+* **MINBATCHTIMEOUT**: when provided with a `t` (expressed in milliseconds) that is greater than 0, the engine will trigger a run even though `MINBATCHSIZE` has not been reached after `t` milliseconds from the time a `MODELEXECUTE` (or the enclosing `DAGRUN`) is enqueued. This only applies to cases where both `BATCHSIZE` and `MINBATCHSIZE` are greater than 0.
+* **INPUTS**: denotes that one or more names of the model's input nodes are following, applicable only for TensorFlow models (specifying INPUTS for other backends will cause an error)
+* **input_count**: a positive number that indicates the number of following input nodes (also applicable only for TensorFlow) 
+* **OUTPUTS**: denotes that one or more names of the model's output nodes are following, applicable only for TensorFlow models (specifying OUTPUTS for other backends will cause an error)
+* **output_count**: a positive number that indicates the number of following input nodes (also applicable only for TensorFlow)
 * **model**: the Protobuf-serialized model. Since Redis supports strings up to 512MB, blobs for very large models need to be chunked, e.g. `BLOB chunk1 chunk2 ...`.
 
 _Return_
@@ -192,10 +194,22 @@ A simple 'OK' string or an error.
 This example shows to set a model 'mymodel' key using the contents of a local file with [`redis-cli`](https://redis.io/topics/cli). Refer to the [Clients Page](clients.md) for additional client choices that are native to your programming language:
 
 ```
-$ cat resnet50.pb | redis-cli -x AI.MODELSET mymodel TF CPU TAG imagenet:5.0 INPUTS images OUTPUTS output BLOB
+$ cat resnet50.pb | redis-cli -x AI.MODELSTORE mymodel TF CPU TAG imagenet:5.0 INPUTS 1 images OUTPUTS 1 output BLOB
 OK
 ```
 
+## AI.MODELSET
+_This command is deprecated and will not be available in future versions. consider using AI.MODELSTORE command instead._
+The **`AI.MODELSET`** command stores a model as the value of a key. The command's arguments and effect are both exactly the same as `AI.MODELEXECUTE` command, except that <input_count> and <output_count> arguments should not be specified for TF backend. 
+
+**Redis API**
+
+```
+AI.MODELSET <key> <backend> <device>
+    [TAG tag] [BATCHSIZE n [MINBATCHSIZE m [MNBATCHTIMEOUT t]]]
+    [INPUTS <name> ...] [OUTPUTS name ...] BLOB <model>
+```
+
 ## AI.MODELGET
 The **`AI.MODELGET`** command returns a model's metadata and blob stored as a key's value.
 
@@ -282,7 +296,49 @@ OK
 !!! note "The `AI.MODELDEL` vis a vis the `DEL` command"
     The `AI.MODELDEL` is equivalent to the [Redis `DEL` command](https://redis.io/commands/del) and should be used in its stead. This ensures compatibility with all deployment options (i.e., stand-alone vs. cluster, OSS vs. Enterprise).
 
+
+## AI.MODELEXECUTE
+The **`AI.MODELEXECUTE`** command runs a model stored as a key's value using its specified backend and device. It accepts one or more input tensors and store output tensors.
+
+The run request is put in a queue and is executed asynchronously by a worker thread. The client that had issued the run request is blocked until the model run is completed. When needed, tensors data is automatically copied to the device prior to execution.
+
+A `TIMEOUT t` argument can be specified to cause a request to be removed from the queue after it sits there `t` milliseconds, meaning that the client won't be interested in the result being computed after that time (`TIMEDOUT` is returned in that case).
+
+!!! warning "Intermediate memory overhead"
+    The execution of models will generate intermediate tensors that are not allocated by the Redis allocator, but by whatever allocator is used in the backends (which may act on main memory or GPU memory, depending on the device), thus not being limited by `maxmemory` configuration settings of Redis.
+
+**Redis API**
+
+```
+AI.MODELEXECUTE <key> INPUTS <input_count> <input> [input ...] OUTPUTS <output_count> <output> [output ...] [TIMEOUT t]
+```
+
+_Arguments_
+
+* **key**: the model's key name
+* **INPUTS**: denotes the beginning of the input tensors keys' list, followed by the number of inputs and one or more key names
+* **input_count**: a positive number that indicates the number of following input keys. 
+* **OUTPUTS**: denotes the beginning of the output tensors keys' list, followed by the number of outputs one or more key names
+* **output_count**: a positive number that indicates the number of output keys to follow.
+* **TIMEOUT**: the time (in ms) after which the client is unblocked and a `TIMEDOUT` string is returned
+
+_Return_
+
+A simple 'OK' string, a simple `TIMEDOUT` string, or an error.
+
+**Examples**
+
+Assuming that running the model that's stored at 'mymodel' with the tensor 'mytensor' as input outputs two tensors - 'classes' and 'predictions', the following command does that:
+
+```
+redis> AI.MODELEXECUTE mymodel INPUTS 1 mytensor OUTPUTS 2 classes predictions
+OK
+```
+
 ## AI.MODELRUN
+
+_This command is deprecated and will not be available in future versions. consider using AI.MODELEXECUTE command instead._   
+
 The **`AI.MODELRUN`** command runs a model stored as a key's value using its specified backend and device. It accepts one or more input tensors and store output tensors.
 
 The run request is put in a queue and is executed asynchronously by a worker thread. The client that had issued the run request is blocked until the model run is completed. When needed, tensors data is automatically copied to the device prior to execution.

docs/intro.md

@@ -193,7 +193,7 @@ RedisAI Tensors are used as inputs and outputs in the execution of models and sc
 ## Loading Models
 A **Model** is a Deep Learning or Machine Learning frozen graph that was generated by some framework. The RedisAI Model data structure represents a DL/ML model that is stored in the database and can be run.
 
-Models, like any other Redis and RedisAI data structures, are identified by keys. A Model's key is created using the [`AI.MODELSET` command](commands.md#aimodelset) and requires the graph payload serialized as protobuf for input.
+Models, like any other Redis and RedisAI data structures, are identified by keys. A Model's key is created using the [`AI.MODELSTORE` command](commands.md#aimodelstore) and requires the graph payload serialized as protobuf for input.
 
 In our examples, we'll use one of the graphs that RedisAI uses in its tests, namely 'graph.pb', which can be downloaded from [here](https://github.com/RedisAI/RedisAI/raw/master/tests/flow/test_data/graph.pb). This graph was created using TensorFlow with [this script](https://github.com/RedisAI/RedisAI/blob/master/tests/flow/test_data/tf-minimal.py).
 
@@ -214,13 +214,13 @@ redis-cli doesn't provide a way to read files' contents, so to load the model wi
 
 ```
 cat graph.pb | docker exec -i redisai redis-cli -x \
-               AI.MODELSET mymodel TF CPU INPUTS a b OUTPUTS c BLOB
+               AI.MODELSTORE mymodel TF CPU INPUTS 2 a b OUTPUTS 1 c BLOB
 ```
 
 ??? example "Example: loading a model from command line"
     ```
     $ cat graph.pb | docker exec -i redisai redis-cli -x \
-               AI.MODELSET mymodel TF CPU INPUTS a b OUTPUTS c BLOB
+               AI.MODELSTORE mymodel TF CPU INPUTS 2 a b OUTPUTS 1 c BLOB
     OK
     ```
 
@@ -230,23 +230,23 @@ cat graph.pb | docker exec -i redisai redis-cli -x \
     * [Redis clients page](https://redis.io/clients)
     * [RedisAI clients page](clients.md)
 
-Like most commands, `AI.MODELSET`'s first argument is a key's name, which is 'mymodel' in the example. The next two arguments are the model's DL/ML backend and the device it will be executed on. 'graph.pb' in the example is a TensorFlow graph and is denoted by `TF` argument. The model will be executed on the CPU as instructed by the `CPU` argument.
+Like most commands, `AI.MODELSTORE`'s first argument is a key's name, which is 'mymodel' in the example. The next two arguments are the model's DL/ML backend and the device it will be executed on. 'graph.pb' in the example is a TensorFlow graph and is denoted by `TF` argument. The model will be executed on the CPU as instructed by the `CPU` argument.
 
-TensorFlow models also require declaring the names of their inputs and outputs. The inputs for 'graph.pb' are called 'a' and 'b', whereas its single output is called 'c'. These names are provided as additional arguments after the 'INPUTS' and 'OUTPUTS' arguments, respectively.
+TensorFlow models also require declaring the names of their inputs and outputs. The inputs for 'graph.pb' are called 'a' and 'b', whereas its single output is called 'c'. These names are provided as additional arguments after indicating 'INPUTS' along with the number of inputs to follow, and 'OUTPUTS' along with the number of outputs to follow, respectively.
 
 ## Running Models
-Once a RedisAI Model key has been set with `AI.MODELSET` it can be run with any Tensor keys from the database as its input. The model's output, after it was executed, is stored in RedisAI Tensors as well.
+Once a RedisAI Model key has been set with `AI.MODELSTORE` it can be run with any Tensor keys from the database as its input. The model's output, after it was executed, is stored in RedisAI Tensors as well.
 
 The model stored at 'mymodel' expects two input tensors so we'll use the previously-create 'tA' and create another input tensor, $\begin{equation*} tB = \begin{bmatrix} 3 \\ 5 \end{bmatrix} \end{equation*}$, with the following command:
 
 ```
 AI.TENSORSET tB FLOAT 2 VALUES 3 5
 ```
 
-The model can now be run with the [`AI.MODELRUN` command](commands.md#aimodelrun) as follows:
+The model can now be run with the [`AI.MODELEXECUTE` command](commands.md#aimodelexecute) as follows:
 
 ```
-AI.MODELRUN mymodel INPUTS tA tB OUTPUTS tResult
+AI.MODELEXECUTE mymodel INPUTS 2 tA tB OUTPUTS 1 tResult
 ```
 
 !!! example "Example: running a model"
@@ -256,11 +256,11 @@ AI.MODELRUN mymodel INPUTS tA tB OUTPUTS tResult
     OK
     127.0.0.1:6379> AI.TENSORSET tB FLOAT 2 VALUES 3 5
     OK
-    127.0.0.1:6379> AI.MODELRUN mymodel INPUTS tA tB OUTPUTS tModel
+    127.0.0.1:6379> AI.MODELEXECUTE mymodel INPUTS 2 tA tB OUTPUTS 1 tModel
     OK
     ```
 
-The first argument to `AI.MODELRUN` is the name of the key at which the RedisAI Model is stored. The names of RedisAI Tensor keys that follow the `INPUTS` argument are used as input for the model. Similarly, following the `OUTPUT` argument are the key names of RedisAI Tensors that the model outputs.
+The first argument to `AI.MODELEXECUTE` is the name of the key at which the RedisAI Model is stored. The names of RedisAI Tensor keys that follow the `INPUTS` and `input_count>` arguments are used as input for the model. Similarly, following the `OUTPUTS` and `output_count>`arguments are the key names of RedisAI Tensors that the model outputs.
 
 The inputs for the example are the tensors stored under the 'tA' and 'tB' keys. Once the model's run had finished, a new RedisAI Tensor key called 'tResult' is created and stores the model's output.
 

src/CMakeLists.txt

@@ -17,6 +17,7 @@ file (GLOB BACKEND_COMMON_SRC
         util/dict.c
         redis_ai_objects/tensor.c
         util/string_utils.c
+        execution/utils.c
         serialization/ai_datatypes.c)
 
 ADD_LIBRARY(redisai_obj OBJECT
@@ -25,8 +26,10 @@ ADD_LIBRARY(redisai_obj OBJECT
         util/string_utils.c
         redisai.c
         execution/command_parser.c
+        execution/deprecated.c
         execution/run_info.c
         execution/background_workers.c
+        execution/utils.c
         config/config.c
         execution/DAG/dag.c
         execution/DAG/dag_parser.c
@@ -43,7 +46,6 @@ ADD_LIBRARY(redisai_obj OBJECT
         rmutil/alloc.c
         rmutil/sds.c
         rmutil/args.c
-        execution/run_info.c
         redis_ai_types/model_type.c
         redis_ai_types/tensor_type.c
         redis_ai_types/script_type.c