Need to be able to set mmap=true for torch.load() for us humble people without very good computers.
That is all.
For those that need a Fix now just make your own ColPaliModel
class CustomColPaliModel(ColPaliModel):
...
def from_index(
...
instance._load_index_with_mmap(index_path)
return instance
}
...
def _load_index_with_mmap(self, index_path: Path):
...
loaded_tensor = torch.load(file, map_location=map_location, mmap=True)
...
If you are using Apple Silicon do NOT use 'mps' when loading the index. Only 'cpu'. mmap does not work otherwise.
Need to be able to set mmap=true for torch.load() for us humble people without very good computers.
That is all.
For those that need a Fix now just make your own ColPaliModel
class CustomColPaliModel(ColPaliModel):
...
def from_index(
...
instance._load_index_with_mmap(index_path)
return instance
}
...
def _load_index_with_mmap(self, index_path: Path):
...
loaded_tensor = torch.load(file, map_location=map_location, mmap=True)
...
If you are using Apple Silicon do NOT use 'mps' when loading the index. Only 'cpu'. mmap does not work otherwise.