Repository navigation
what type of data that the model should feed in ? #432
Description
hello
im always worry about the type of data should give to the model .
so for mlp is should be always flatten data right or can be other shape too ?
also what is better the normalization or standardization ? is cannot be raw data right ?
also we cant feed json data directly or with some encoder ?
thank you for your time i appreciate you answer
Activity
puyuan1996 commentedon Oct 19, 2025
Should data for an MLP always be flattened?
Yes, a standard Multi-Layer Perceptron (MLP) or fully-connected network requires a 1D vector (i.e., flattened data) as input. However, networks like CNNs (for images) or Transformers (for text) can process multi-dimensional or sequential data directly, only flattening the features when connecting to an MLP layer.
Normalization vs. Standardization? Can I use raw data?
Do not use raw data, as varying feature scales will severely impact training. Both normalization (scaling to [0, 1]) and standardization (mean 0, variance 1) are essential preprocessing steps with no absolute "better" one. A common best practice in deep learning is to normalize input images and use Batch Normalization or Layer Normalization inside the network.
Can we feed JSON data directly?
Directly feeding JSON data is not currently supported, but you can adapt it. Models only understand numerical tensors. You must write an encoder to parse the JSON and convert its contents (key-value pairs, text, categories, etc.) into a fixed-length numerical vector. This process is analogous to how the Tokenizer in UniZero model converts text into a sequence of numerical IDs.
procodingjob commentedon Oct 24, 2025
thank you so much for you answer , that mean if we have only one feature raw can work but multiple features columns can impact the neural network .
is MLP catch the data steps as sequences , or is better to put the sequences on OBS even its a flatten data , like concatenate the arrays ? and how to separate with them to help ai understand what we give him . and what is better to append or insert i mean have the last sequence at the end or at the first .
and other questions please , if we have multiple type of data can we apply the normalization and standardization to deferent type of data and put it all in one observation ? i mean having normalization and standardization in one flatten array ?
thank you
procodingjob commentedon Oct 26, 2025
and other question please , there is any way that can we give to agent the next step action to choice , instead of agent take a random action to learn , like an other value pass via info on step or reset to describe the next action agent should take. to learn directly the good actions is that possible ?
puyuan1996 commentedon Nov 17, 2025
Hello @procodingjob, these are excellent and critically important questions for practical application!
1. Regarding Sequential Data and MLP Handling
First, a small clarification on your summary of the previous answer: even with just a single feature, if its numerical range is very large or unbounded (e.g., from 0 to 1,000,000), not preprocessing it will still harm the model's training stability and convergence speed. Therefore, preprocessing is generally necessary regardless of the number of features.
Now, returning to your new questions about sequential data:
Can an MLP Capture Sequential Relationships?
A standard MLP, by itself, has no memory and cannot understand the temporal order within input data. When you feed a flattened data point from one time step into an MLP, it treats it as a single, static vector. If you feed new data from the next time step, the MLP will have no memory of the previous step.
How to Make an MLP "See" a Sequence?
Your intuition is exactly right! The most common method is to concatenate observations from several consecutive time steps into a single, longer vector, which is then used as a single input to the MLP. This technique is often called "Frame Stacking" or using a "History Window."
For example, if your environment's observation at time step
tisobs_t, you can construct a new inputinput_t = [obs_t, obs_{t-1}, obs_{t-2}, ...]. This way, the MLP can simultaneously "see" the current state and the states from the past few steps, allowing it to indirectly learn about dynamics and temporal correlations in the data.How to Separate and Order the Data?
[1, 2, 3, 4]and[5, 6, 7, 8], results in a single vector of length 8:[1, 2, 3, 4, 5, 6, 7, 8]. The neural network will automatically learn through its weights to distinguish different parts of this long vector (e.g., the first 4 elements represent the current state, the next 4 represent the previous state). The key is to maintain this structure consistently throughout the entire training process.[obs_t, obs_{t-1}, ...]) or at the end ([..., obs_{t-1}, obs_t]) are both viable options. The neural network is flexible enough to adapt to either fixed order. In practice, placing the newest data at the beginning is perhaps slightly more common, as it keeps the most relevant information (the current state) at a fixed position in the input vector (starting at index 0). However, there is no absolute superiority of one method over the other, as long as you remain consistent.2. Regarding Mixed Data Preprocessing
Absolutely, and this is a very standard and recommended practice.
Different features often have different physical meanings, units, and numerical distributions. Applying the most suitable preprocessing method to each is crucial.
For example: Assume your observation contains three types of information:
After preprocessing, you concatenate these three processed data parts into one large, flattened array and feed it into the model. The model doesn't "care" whether a part was normalized or standardized; it only sees an input vector with numerical ranges that are conducive to learning and learns patterns from it.
So, mixing normalized and standardized data within a single flattened array is entirely correct and common.
3. Regarding Guided Agent Learning (Imitation Learning)
Yes, the idea you're describing is a very important branch of reinforcement learning known as Imitation Learning (IL) or Behavioral Cloning (BC).
Basic Idea:
The core idea is to leverage a dataset provided by an "expert" (like a human player or an already trained model) before letting the agent explore on its own (which can be very inefficient). This dataset consists of a series of <state, expert_action> pairs.
We transform this problem into a supervised learning problem: the model's (i.e., the agent's policy network) input is the
state, and its goal is to output an action that is as close as possible to theexpert_action.Application in LightZero:
This method is known as "Pre-training." You can first pre-train your model on expert data using behavioral cloning to give it a reasonably good initial policy. Then, starting from this pre-trained model, you can use a reinforcement learning algorithm (like AlphaZero or MuZero) for "Fine-tuning," allowing it to further improve through self-play. This "imitate first, then surpass" paradigm is very effective in many complex tasks. The LightZero framework is designed to support, or can be extended to support, this workflow.
I hope these detailed explanations help you better understand these concepts! If you have any other questions, please feel free to ask. (Part of the answer was assisted by Gemini 2.5 Pro.)