我们收集了一个以用户需求为中心的电商对话式推荐数据集（U-NEED）。
We collect a user needs-centric E-commerce conversational recommendation dataset (U-NEED).
U-NEED包含了7,698个细粒度标注的售前对话，333,879个用户行为和332,148条商品知识元组。
U-NEED consists of 7,698 fine-grained annotated pre-sales dialogues, 333,879 user behaviors and 332,148 product knowledge tuples.
对于售前对话的每一条语句，我们雇佣了专业的众包平台来标注：说话人的动作，语句涉及的属性和语句中推荐的商品。
For each utterance of pre-sales dialogue, we hire a professional crowdsourcing platform to annotate the action of the speaker, the attributes involved, and the recommended products.
| Category | Model | Precision | Recall | F1 |
|---|---|---|---|---|
| All | Bert+BiLSTM+CRF | 68.92% | 68.75% | 0.6884 |
| Bert+CRF | 66.88% | 65.30% | 0.6608 | |
| Bert | 45.49% | 56.52% | 0.5041 | |
| Beauty | Bert+BiLSTM+CRF | 72.82% | 74.81% | 0.7380 |
| Bert+CRF | 67.31% | 68.02% | 0.6766 | |
| Bert | 53.55% | 62.84% | 0.5782 | |
| Fashion | Bert+BiLSTM+CRF | 65.89% | 70.71% | 0.6822 |
| Bert+CRF | 60.16% | 66.61% | 0.6322 | |
| Bert | 46.45% | 58.39% | 0.5174 | |
| Phones | Bert+BiLSTM+CRF | 67.01% | 69.90% | 0.6843 |
| Bert+CRF | 56.20% | 59.23% | 0.5768 | |
| Bert | 42.12% | 53.84% | 0.4726 | |
| Electronic | Bert+BiLSTM+CRF | 65.48% | 67.71% | 66.58% |
| Bert+CRF | 62.12% | 61.55% | 0.6183 | |
| Bert | 39.81% | 49.40% | 0.4409 | |
| Shoes | Bert+BiLSTM+CRF | 78.70% | 81.01% | 0.7984 |
| Bert+CRF | 73.02% | 77.03% | 0.7497 | |
| Bert | 58.51% | 70.20% | 0.6382 |
| Category | Model | Precision | Recall | F1 |
|---|---|---|---|---|
| All | DiaMultiClass | 0.3222 | 0.4966 | 0.3662 |
| DiaSeq | 0.3555 | 0.2966 | 0.3153 | |
| Beauty | DiaMultiClass | 0.4037 | 0.7228 | 0.3662 |
| DiaSeq | 0.4761 | 0.4272 | 0.4424 | |
| Fashion | DiaMultiClass | 0.2711 | 0.3488 | 0.2918 |
| DiaSeq | 0.1525 | 0.1271 | 0.1355 | |
| Phones | DiaMultiClass | 0.4534 | 0.5212 | 0.4585 |
| DiaSeq | 0.4414 | 0.3789 | 0.3966 | |
| Electronic | DiaMultiClass | 0.2567 | 0.3657 | 0.2851 |
| DiaSeq | 0.2420 | 0.1736 | 0.1891 | |
| Shoes | DiaMultiClass | 0.3361 | 0.4131 | 0.3423 |
| DiaSeq | 0.3992 | 0.3305 | 0.3498 |
| Category | Model | Hits@10 | Hits@50 | NDCG@10 | NDCG@50 | MRR@10 | MRR@50 |
|---|---|---|---|---|---|---|---|
| All | Bert | 0.1593 | 0.331 | 0.0818 | 0.1192 | 0.0582 | 0.066 |
| SASRec | 0.14 | 0.2747 | 0.0725 | 0.1022 | 0.0522 | 0.0585 | |
| TGCRS | 0.147 | 0.25 | 0.0809 | 0.1036 | 0.0606 | 0.0655 | |
| Beauty | Bert | 0.2985 | 0.3938 | 0.1842 | 0.2057 | 0.1484 | 0.1532 |
| SASRec | 0.1631 | 0.3169 | 0.0726 | 0.1067 | 0.0449 | 0.0523 | |
| TGCRS | 0.2831 | 0.3723 | 0.1722 | 0.1914 | 0.1374 | 0.1413 | |
| Fashion | Bert | 0.1348 | 0.1489 | 0.0854 | 0.0885 | 0.0697 | 0.0703 |
| SASRec | 0.0496 | 0.0816 | 0.026 | 0.0333 | 0.0188 | 0.0204 | |
| TGCRS | 0.1099 | 0.1241 | 0.0755 | 0.0786 | 0.065 | 0.0657 | |
| Phones | Bert | 0.4275 | 0.7174 | 0.2488 | 0.3138 | 0.1943 | 0.2086 |
| SASRec | 0.4094 | 0.7065 | 0.2134 | 0.2808 | 0.1546 | 0.1698 | |
| TGCRS | 0.5942 | 0.7681 | 0.3392 | 0.3791 | 0.2609 | 0.2702 | |
| Electronic | Bert | 0.2576 | 0.4333 | 0.1484 | 0.1864 | 0.1145 | 0.1222 |
| SASRec | 0.1758 | 0.2788 | 0.1067 | 0.1283 | 0.0849 | 0.089 | |
| TGCRS | 0.2818 | 0.397 | 0.169 | 0.1949 | 0.1344 | 0.1402 | |
| Shoes | Bert | 0.1014 | 0.255 | 0.0473 | 0.0813 | 0.0312 | 0.0384 |
| SASRec | 0.0691 | 0.1674 | 0.0388 | 0.0602 | 0.0296 | 0.0341 | |
| TGCRS | 0.1521 | 0.255 | 0.083 | 0.1058 | 0.0618 | 0.0668 |
| Category | Model | dist@1 | dist@2 | dist@3 | dist@4 | bleu@1 | bleu@2 | bleu@3 | bleu@4 | Info | Rel |
|---|---|---|---|---|---|---|---|---|---|---|---|
| All | GPT-2 | 0.0284 | 0.0624 | 0.1780 | 0.2905 | 0.0688 | 0.0276 | 0.0166 | 0.0136 | 0.5700 | 0.4267 |
| Transformer | 0.01462 | 0.05366 | 0.1563 | 0.2806 | 0.1138 | 0.03715 | 0.02037 | 0.01359 | 1.1567 | 0.8800 | |
| KBRD | 0.01173 | 0.04061 | 0.1259 | 0.2233 | 0.1253 | 0.04067 | 0.02528 | 0.01879 | 1.1367 | 0.9167 | |
| NTRD | 0.1485 | 0.1942 | 0.2277 | 0.2489 | 0.0443 | 0.0082 | 0.0028 | 0.0016 | 1.0033 | 0.9900 | |
| Beauty | GPT-2 | 0.0581 | 0.1250 | 0.2555 | 0.3811 | 0.0610 | 0.0176 | 0.0054 | 0.0025 | 0.6433 | 0.2767 |
| Transformer | 0.03654 | 0.09977 | 0.2524 | 0.3714 | 0.09455 | 0.02516 | 0.01466 | 0.009815 | 1.2767 | 0.5267 | |
| KBRD | 0.03466 | 0.08759 | 0.1977 | 0.2808 | 0.1097 | 0.0325 | 0.01999 | 0.01389 | 1.2233 | 0.6133 | |
| NTRD | 0.1259 | 0.2277 | 0.2963 | 0.3275 | 0.0439 | 0.0083 | 0.0042 | 0.0030 | 1.1500 | 0.6867 | |
| Phones | GPT-2 | 0.0497 | 0.1099 | 0.2204 | 0.3266 | 0.1110 | 0.0460 | 0.0248 | 0.0173 | 0.7633 | 0.4700 |
| Transformer | 0.04254 | 0.1209 | 0.2853 | 0.4037 | 0.1279 | 0.04393 | 0.02546 | 0.01471 | 1.2267 | 0.9467 | |
| KBRD | 0.0506 | 0.1359 | 0.3017 | 0.4157 | 0.1418 | 0.04371 | 0.0204 | 0.008948 | 1.0900 | 0.9867 | |
| NTRD | 0.1614 | 0.2666 | 0.3190 | 0.3775 | 0.0560 | 0.0163 | 0.0102 | 0.0072 | 1.0400 | 1.0567 | |
| Shoes | GPT-2 | 0.0522 | 0.1171 | 0.228 | 0.3357 | 0.0803 | 0.0405 | 0.0270 | 0.0202 | 0.5000 | 0.3767 |
| Transformer | 0.0319 | 0.07772 | 0.1975 | 0.3718 | 0.117 | 0.05629 | 0.04072 | 0.03035 | 1.0400 | 0.9600 | |
| KBRD | 0.03508 | 0.08901 | 0.2224 | 0.4051 | 0.1358 | 0.0685 | 0.04701 | 0.03219 | 1.0933 | 1.0467 | |
| NTRD | 0.2220 | 0.4036 | 0.4499 | 0.4635 | 0.0432 | 0.0135 | 0.0069 | 0.0041 | 1.0100 | 0.9600 |
| Category | Model | PCC | SCC | Cos |
|---|---|---|---|---|
| All | DEB | 0.1617 | 0.1864 | 0.9212 |
| P-value | <6e-06 | <1e-07 | - | |
| Bert-RUBER | 0.0742 | 0.1092 | 0.9214 | |
| P-value | <0.0398 | <0.0024 | - | |
| Beauty | DEB | 0.1642 | 0.1628 | 0.9327 |
| P-value | <0.0299 | <0.0313 | - | |
| Bert-RUBER | 0.0901 | 0.1133 | 0.9218 | |
| P-value | -0.0126 | <0.0017 | - | |
| Phones | DEB | 0.2678 | 0.2815 | 0.9366 |
| P-value | <0.0015 | <0.0008 | - | |
| Bert-RUBER | 0.0900 | 0.1141 | 0.9218 | |
| P-value | <0.0126 | <0.0015 | - | |
| Shoes | DEB | 0.1504 | 0.1963 | 0.9097 |
| P-value | <0.0416 | <0.0076 | - | |
| Bert-RUBER | 0.0916 | 0.1157 | 0.9219 | |
| P-value | <0.0111 | <0.0013 | - |