AttributeError: 'AVHubertSeq2Seq' object has no attribute 'prompt_init'

[ouwen18]

Dear author, hello. When I fine-tune the pre-trained model using voice and attempt few-shot fine-tuning with visual information, I encountered the following error. Is it possible that there is a lack of definition for this class in the code?
Traceback (most recent call last):
File "/home/aa/anaconda3/envs/openSR/bin/fairseq-hydra-train", line 8, in
sys.exit(cli_main())
File "/home/aa/OpenSR/fairseq/fairseq_cli/hydra_train.py", line 76, in cli_main
hydra_main()
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
raise ex
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/internal/hydra.py", line 107, in run
return run_job(
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/hydra/core/utils.py", line 129, in run_job
ret.return_value = task_function(task_cfg)
File "/home/aa/OpenSR/fairseq/fairseq_cli/hydra_train.py", line 45, in hydra_main
distributed_utils.call_main(cfg, pre_main)
File "/home/aa/OpenSR/fairseq/fairseq/distributed/utils.py", line 369, in call_main
main(cfg, **kwargs)
File "/home/aa/OpenSR/fairseq/fairseq_cli/train.py", line 97, in main
model = task.build_model(cfg.model)
File "/home/aa/OpenSR/fairseq/fairseq/tasks/fairseq_task.py", line 325, in build_model
model = models.build_model(cfg, self)
File "/home/aa/OpenSR/fairseq/fairseq/models/init.py", line 96, in build_model
return model.build_model(cfg, task)
File "/home/aa/OpenSR/opensr/hubert_asr.py", line 489, in build_model
encoder = HubertEncoderWrapper(encoder, cfg.prompting, cfg.prompt_strategy)
File "/home/aa/OpenSR/opensr/hubert_asr.py", line 391, in init
self.w2v_model.prompt_init(strategy=strategy)
File "/home/aa/anaconda3/envs/openSR/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'AVHubertSeq2Seq' object has no attribute 'prompt_init'

[Exgc]

Hello, since you didn’t provide the command line earlier, we were unable to reproduce your error. Recently, some researchers reported similar issues to us and pinpointed the cause of the problem.

The following command:

fairseq-hydra-train --config-dir /path/to/conf/ --config-name prompt/large_vox_base_{10,20,50,100}.yaml
task.data=/path/to/data task.label_dir=/path/to/label
task.tokenizer_bpe_model=/path/to/tokenizer model.w2v_path=/path/to/checkpoint
hydra.run.dir=/path/to/experiment/few-shot common.user_dir=pwd

In this case, the w2v_path should point to the pretrained checkpoint(i.e,large_vox_iter5.pt), not the finetuned checkpoint.