Skip to content

Pytorch implementation of EMNLP 2021 paper "TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph "

Notifications You must be signed in to change notification settings

Pobby321/TransferNet

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TransferNet

Pytorch implementation of EMNLP 2021 paper

TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph
Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, Hanwang Zhang

We perform transparent multi-hop reasoning over relation graphs of label form (i.e., knowledge graph) and text form. This is an example:


If you find this code useful in your research, please cite

@inproceedings{shi2021transfernet,
  title={TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph},
  author={Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, Hanwang Zhang},
  booktitle={EMNLP},
  year={2021}
}

dependencies

Prepare Datasets

  • MetaQA, we only use its vanilla version.
  • MovieQA, we need its knowledge_source/wiki.txt as the text corpus for our MetaQA-Text experiments. Copy the file into the folder of MetaQA, and put it together with kb.txt. The files of MetaQA should be something like
MetaQA
+-- kb
|   +-- kb.txt
|   +-- wiki.txt
+-- 1-hop
|   +-- vanilla
|   |   +-- qa_train.txt
|   |   +-- qa_dev.txt
|   |   +-- qa_test.txt
+-- 2-hop
+-- 3-hop
python pickle_glove.py --txt </path/to/840B.300d.txt> --pt </output/file/name>

Experiments

MetaQA-KB

  1. Preprocess
python -m MetaQA-KB.preprocess --input_dir <PATH/TO/METAQA> --output_dir <PATH/TO/PROCESSED/FILES>
  1. Train
python -m MetaQA-KB.train --glove_pt <PATH/TO/GLOVE/PICKLE> --input_dir <PATH/TO/PROCESSED/FILES> --save_dir <PATH/TO/CHECKPOINT>
  1. Predict on the test set
python -m MetaQA-KB.predict --input_dir <PATH/TO/PROCESSED/FILES> --ckpt <PATH/TO/CHECKPOINT> --mode test
  1. Visualize the reasoning process. It will enter an IPython environment after showing the information of each sample. You can print more variables that you are insterested in. To stop the process, you need to quit the IPython by Ctrl+D and then kill the loop by Ctrl+C immediately.
python -m MetaQA-KB.predict --input_dir <PATH/TO/PROCESSED/FILES> --ckpt <PATH/TO/CHECKPOINT> --mode vis

MetaQA-Text

  1. Preprocess
python -m MetaQA-Text.preprocess --input_dir <PATH/TO/METAQA> --output_dir <PATH/TO/PROCESSED/FILES>
  1. Train
python -m MetaQA-Text.train --glove_pt <PATH/TO/GLOVE/PICKLE> --input_dir <PATH/TO/PROCESSED/FILES> --save_dir <PATH/TO/CHECKPOINT>

The scripts for inference and visualization are the same as MetaQA-KB. Just change the python module to MetaQA-Text.predict.

MetaQA-Text + 50% KB

  1. Preprocess
python -m MetaQA-Text.preprocess --input_dir <PATH/TO/METAQA> --output_dir <PATH/TO/PROCESSED/FILES> --kb_ratio 0.5
  1. Train, it needs more active paths than MetaQA-Text
python -m MetaQA-Text.train --input_dir <PATH/TO/PROCESSED/FILES> --save_dir <PATH/TO/CHECKPOINT> --max_active 800 --batch_size 32

The scripts for inference and visualization are the same as MetaQA-Text.

WebQSP

WebQSP does not need preprocess. We can directly start the training:

python -m WebQSP.train --input_dir <PATH/TO/UNZIPPED/DATA> --save_dir <PATH/TO/CHECKPOINT>

ComplexWebQuestions

Similar to WebQSP, CWQ does not need preprocess. We can directly start the training:

python -m CompWebQ.train --input_dir <PATH/TO/UNZIPPED/DATA> --save_dir <PATH/TO/CHECKPOINT>

About

Pytorch implementation of EMNLP 2021 paper "TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph "

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%