And 20 Long Tasks executed as a sequence of skills.

The dataset provides the following modalities:

• Proprioception
• Tactile Sensing

The dataset files are organised as following:

DexSkill_dataset
    └─ dataset / Long-horizon task dataset
         └── data_0.pt
         └── ...
         └── data_i.pt
         │   ├── state_input
         │   ├── state_output
         │   ├── feature_input
         │   ├── feature_output
         │   ├── label
    

There are 60 dataset files for the training, each consisting of a batch size of 256 with data shuffled. The dataset is saved in a dictionary style.

• data['state_input'] contains the raw haptic data, including the end-effector state, filtered tactile information, filtered contact indicators, and the AH joint state.
• data['feature_input'] includes proposed features while excluding the raw haptic data.
• data['state_output'] and data['features_output'] is the proposed feature at the next timestep, which is used to train the auto-regressive autoencoder.
• data['label'] includes the skill name for the recorded task.

The .pt files located within the /DexSkill_dataset/dataset directory encompass a comprehensive collection of recorded demonstrations across 20 primitive skills. Additionally, within the /DexSkill_dataset/LH_dataset folder, each .pt file correspond to a specific long-horizon manipulation task, with no shuffling involved to preserve the time-series sequence of these tasks.

All trained policies, including those of our framework and comparative works, are inside the trained_policy folder. Furthermore, the json_file within this dataset provides human-labeled task segmentation for all long-horizon tasks, serving as a ground truth.

The demo code for load the dataset and train the classifier is in the file /code/train_classifier.py