[API Doc] Syntax-related issues and other possible problems in Orca API doc

[Oscilloscope98]

Here lists the problems occurred in Orca API caused by wrong rst syntax, and some other possible mistakes.

General Suggestions

• For value options written in "value" or 'value' or even value, would it be clearer to be emphasized through inline code as ``value``?
  • example api doc without inline code highlight (our common cases)
    
  • example api doc with inline code highlight
    
    We could also use inline code for data type, method, etc.
• Would it be more user-friendly to explain the return value of each method, even if the return value is None?
  • Currently there are cases where returns field is empty, or there is no returns field, or there is a description "None" for the returns field, which has bad consistency.
    
• For methods not supported in API docs, would it be better to use for example a warning box to emphasize them or differentiate them from other normal API?
  • example with plain text
    
  • example with warning box
    

Detailed issue lists

Orca Context

orca.common.init_orca_context

• method bigdl.orca.common.init_orca_context
  • parameter runtime: duplicated explanation
  • parameter cluster_mode: spark-submit and bigdl-submit are in wrong rst inline code syntax. Use ``inline code``.

Orca Data

orca.data.XShards

• class bigdl.orca.data.XShards method partition
  • parameter num_shards: wrong field list syntax, failed to be recognized as a parameter, should check indentation.
  • returns: wrong field list syntax, failed to be recognized as a parameter, should check indentation.

orca.data.pandas

• method bigdl.orca.data.pandas.preprocessing.read_csv

  • parameter kwargs: wrong field list syntax, failed to be recognized as a parameter, should check indentation.
  • returns: wrong field list syntax, failed to be recognized as a return field, should check indentation.

• method bigdl.orca.data.pandas.preprocessing.read_json

  • parameter kwargs: wrong field list syntax, failed to be recognized as a parameter, should check indentation.
  • returns: wrong field list syntax, failed to be recognized as a return field, should check indentation.

• method bigdl.orca.data.pandas.preprocessing.read_file_spark

  • No description for this method, including explanation for parameters

• method bigdl.orca.data.pandas.preprocessing.read_parquet

  • parameter columns, schema, options: wrong field list syntax, failed to be recognized as a parameter, should check indentation.
  • returns: wrong field list syntax, failed to be recognized as a return field, should check indentation.

Orca Learn

orca.learn.bigdl.estimator

• class [bigdl.orca.learn.bigdl.estimator.Estimator](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Orca/orca.html#bigdl.orca.learn.bigdl.estimator.Estimator]

  • No description for this class

• class bigdl.orca.learn.bigdl.estimator.Estimator method from_bigdl

  • parameter feature_preprocessing: fit and predict are in wrong rst inline code syntax. Use ``inline code``.
  • parameter label_preprocessing: fit and predict are in wrong rst inline code syntax. Use ``inline code``.
  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method fit

  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method evaluate

  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method save

  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method load

  • parameter feature_preprocessing: fit and predict are in wrong rst inline code syntax. Use ``inline code``.
  • parameter label_preprocessing: fit and predict are in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method load_orca_checkpoint

  • description: version and perfix are in wrong rst inline code syntax. Use ``inline code``.
  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method clear_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method set_constant_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method set_l2_norm_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method get_train_summary

  • returns: the description for return values is written in method description. Would be clearer to use return field list for it.

• class bigdl.orca.learn.bigdl.estimator.BigDLEstimator method get_validation_summary

  • returns: the description for return values is written in method description. Would be clearer to use return field list for it.

orca.learn.tf.estimator

• class [bigdl.orca.learn.tf.estimator.Estimator](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Orca/orca.html#bigdl.orca.learn.tf.estimator.Estimator]

  • No description for this class

• class bigdl.orca.learn.tf.estimator.Estimator method save

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.Estimator method load

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.Estimator method clear_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.Estimator method set_constant_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.Estimator method set_l2_norm_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.Estimator method get_train_summary

  • returns: the description for return values is written in method description. Would be clearer to use return field list for it.

• class bigdl.orca.learn.tf.estimator.Estimator method get_validation_summary

  • returns: the description for return values is written in method description. Would be clearer to use return field list for it.

• class bigdl.orca.learn.tf.estimator.Estimator method save_keras_weights

  • parameter save_format: filepath, save_format and None are in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.tf.estimator.Estimator method load_orca_checkpoint

  • description: version is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.tf.estimator.Estimator method from_graph

  • parameter metrics: its order in parameter description list seems to be wrong, not the same as definition

• method bigdl.orca.learn.tf.estimator.is_tf_data_dataset

  • No description for this method, including explanation for parameters

• method bigdl.orca.learn.tf.estimator.to_dataset

  • No description for this method, including explanation for parameters

• method bigdl.orca.learn.tf.estimator.save_model_dir

  • No description for this method, including explanation for parameters

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator method fit

  • parameter auto_shard_files, session_config, feed_dict, checkpoint_trigger: their order in parameter description list seems to be wrong, not the same as definition

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator method load_tf_checkpoint

  • parameter path: wrong field list syntax, failed to be recognized as a parameter, should check indentation.

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator method save

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator method load

  • parameter model_path: wrong field list syntax, failed to be recognized as a parameter, should check indentation.
  • returns: wrong field list syntax, failed to be recognized as a return field, should check indentation.
  • returns: empty description

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator method set_constant_gradient_clipping

  • parameter min: missing description
  • parameter max: missing description

• class bigdl.orca.learn.tf.estimator.TensorFlowEstimator method set_l2_norm_gradient_clipping

  • parameter clip_norm: missing description

• class bigdl.orca.learn.tf.estimator.KerasEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.tf.estimator.KerasEstimator method save

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.KerasEstimator method load

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.KerasEstimator method clear_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.KerasEstimator method set_constant_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.KerasEstimator method set_l2_norm_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.tf.estimator.KerasEstimator method save_keras_weights

  • parameter save_keras_weights: filepath, save_format, and None are in wrong rst inline code syntax. Use ``inline code``.

orca.learn.tf2.estimator

• class bigdl.orca.learn.tf2.estimator.Estimator

  • No description for this class

• class bigdl.orca.learn.tf2.estimator.Estimator method from_keras

  • parameter model_creator: config is in wrong rst inline code syntax. Use ``inline code``.
  • parameter config: fit_config, model.fit(data, **fit_config), evaluate_config, and model.evaluate are in wrong rst inline code syntax. Use ``inline code``.
  • parameter compile_args_creator: config is in wrong rst inline code syntax. Use ``inline code``.
  • parameter backend: ray is in wrong rst inline code syntax. Use ``inline code``.
  • parameter model_dir: wrong field list syntax, a part of its descriptions is not recognized, should check indentation.

• class bigdl.orca.learn.tf2.estimator.Estimator method latest_checkpoint

  • No description for this method, including explanation for parameters

• method bigdl.orca.learn.tf2.estimator.make_data_creator

  • No description for this method, including explanation for parameters

• method bigdl.orca.learn.tf2.estimator.data_length

  • No description for this method, including explanation for parameters

orca.learn.tf2.tf2_ray_estimator

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method fit

  • parameter steps_per_epoch: steps_pre_epoch, None are in wrong rst inline code syntax. Use ``inline code``.
  • parameter validation_freq: collections_abc.Container, validation_freq=2, and validation_freq=[1, 2, 10] are in wrong rst inline code syntax. Use ``inline code``.
  • parameter data_config: output_signature, tf.data.Dataset.from_generator, and tf.TypeSpec are in wrong rst inline code syntax. Use ``inline code``.
  • returns: empty description

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method evaluate

  • parameter data_config: output_signature, tf.data.Dataset.from_generator, and tf.TypeSpec are in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method process_ray_dataset

  • No description for this method, including explanation for parameters

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method predict

  • returns: empty description

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method get_model

  • parameter sample_input: missing description

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method load_checkpoint

  • parameter kwargs: missing description

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method save

  • parameter save_format: tf and h5 are in wrong rst inline code syntax. Use ``inline code``.
  • parameter signatures: signatures and tf.saved_model.save are in wrong rst inline code syntax. Use ``inline code``.
  • parameter options: tf.saved_model.SaveOptions is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method load

  • description: estimator.save() is in wrong rst inline code syntax. Use ``inline code``.
  • parameter options: tf.saved_model.LoadOptions is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method save_weights

  • parameter filepath, overwrite, save_format, options: wrong field list syntax, parameter field list should start with :param parameter_name:
  • returns: empty description

• class bigdl.orca.learn.tf2.ray_estimator.TensorFlow2Estimator method load_weights

  • parameter filepath, by_name, skip_mismatcht, options: wrong field list syntax, parameter field list should start with :param parameter_name:`
  • returns: empty description

orca.learn.tf2.tf2_spark_estimator

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator method fit

  • parameter data: problematic field list syntax. As the first item in the field list, it should have at list one blank line before the previous texts
  • parameter initial_epoch, steps_per_epoch, validation_steps, validation_freq, data_config, feature_cols, and label_cols: missing description
  • returns: empty description

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator method evaluate

  • parameter data: problematic field list syntax. As the first item in the field list, it should have at list one blank line before the previous texts
  • parameter validation_data and class_weight are not in the method definition, but occur in the description
  • parameter num_steps, sample_weight, data_config, feature_cols, and label_cols: missing description

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator method predict

  • parameter data: problematic field list syntax. As the first item in the field list, it should have at list one blank line before the previous texts
  • returns: empty description

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator method save_weights

  • parameter filepath, overwrite and save_format: wrong field list syntax, failed to be recognized as a parameter, should check indentation.

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator method save

  • parameter signatures: signatures and tf.saved_model.save are in wrong rst inline code syntax. Use ``inline code``.
  • parameter options: tf.saved_model.SaveOptions is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.tf2.pyspark_estimator.SparkTFEstimator method load

  • parameter options: tf.saved_model.LoadOptions is in wrong rst inline code syntax. Use ``inline code``.

orca.learn.pytorch.estimator

• class bigdl.orca.learn.pytorch.estimator.Estimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.pytorch.estimator.Estimator method from_torch

  • parameter training_operator_cls and initialization_hook: missing description
  • parameter scheduler_creator: horovod and ray are in wrong rst inline code syntax. Use ``inline code``.
  • parameter use_tqdm: horovod and ray are in wrong rst inline code syntax. Use ``inline code``.
  • parameter workers_per_node: horovod and ray are in wrong rst inline code syntax. Use ``inline code``.
  • parameter model_dir: bigdl and spark are in wrong rst inline code syntax. Use ``inline code``.
  • parameter backend: bigdl is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.pytorch.estimator.Estimator method latest_checkpoint

  • No description for this method, including explanation for parameters

orca.learn.pytorch.pytorch_ray_estimator

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method fit

  • description: TrainingOperator.train_epoch() is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method predict

  • parameter feature_cols and profile: their order in parameter description list seem to be wrong, not the same as definition

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method evaluate

  • description: TrainingOperator.validate() is in wrong rst inline code syntax. Use ``inline code``.
  • parameter num_steps: TrainingOperator.validate_batch is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method save

  • returns: empty description

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method shutdown

  • parameter force: missing description
  • returns: empty description

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method get_state_dict

  • No description for this method

• class bigdl.orca.learn.pytorch.pytorch_ray_estimator.PyTorchRayEstimator method load_state_dict

  • No description for this method, including explanation for parameters

orca.learn.pytorch.pytorch_spark_estimator

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method load

  • description: state_dict is in wrong rst inline code syntax. Use ``inline code``.
  • returns: empty description

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method load_orca_checkpoint

  • description: version and perfix are in wrong rst inline code syntax. Use ``inline code``.
  • returns: empty description

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method get_train_summary

  • the description for return values is written in method description. Would be clearer to use return field list for it.

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method get_validation_summary

  • the description for return values is written in method description. Would be clearer to use return field list for it.

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method clear_gradient_clipping

  • returns: empty description

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method set_l2_norm_gradient_clipping

  • returns: empty description

orca.learn.openvino.estimator

• class bigdl.orca.learn.openvino.estimator.Estimator

  • No description for this class

• class bigdl.orca.learn.openvino.estimator.OpenvinoEstimator

  • No description for this class, including explanation for parameters

• class bigdl.orca.learn.openvino.estimator.predict

  • parameter batch_size: its order in parameter description list seems to be wrong, not the same as definition
  • parameter inputs_cols: inputs is in wrong rst inline code syntax. Use ``inline code``.

• class bigdl.orca.learn.openvino.estimator.predict method load

  • returns: empty description

Orca AutoML

orca.automl.auto_estimator

• class bigdl.orca.automl.auto_estimator.AutoEstimator
  • Missing class description and parameter explanation

orca.automl.metrics

• method bigdl.orca.automl.metrics.Accuracy
  • parameter multioutput: missing description

orca.automl.auto_xgb

• class bigdl.orca.automl.xgboost.auto_xgb.AutoXGBClassifier

  • parameter xgb_configs: the hyperlink would be better to be added with link text, that is written in `text <example/link>`_

• class bigdl.orca.automl.xgboost.auto_xgb.AutoXGBRegressor

  • parameter xgb_configs: the hyperlink would be better to be added with link text, that is written in `text <example/link>`_

Other possible issues

• The section title "orca.init_orca_context" in Orca Context seems to be wrong. Should it be "orca.common.init_orca_context"?

• bigdl.orca.learn.bigdl.estimator.BigDLEstimator method evaluate

  • Default value of feature_cols and label_cols seem to be wrong in description

• class bigdl.orca.learn.pytorch.pytorch_spark_estimator.PyTorchSparkEstimator method fit

• Default value of batch_size is None in definition, but it is 32 in description. Is there something wrong?

• The section title "orca.automl.auto_xgb" in Orca AutoML seems to be wrong. Should it be "orca.automl.xgboost.auto_xgb"?