Integrate HICR as hicr engine into LPF - #23

Open

KADichev wants to merge 44 commits into

Commits on Sep 20, 2023

Separate the ibv_post_send and ibv_poll_cq into different functions, so that these could be assigned to different LPF functions (e.g., trigger send early by moving ibv_post_send calls into IBVerbs:…
KADichev
committed
0187aab

Commits on Sep 25, 2023

Extended LPF to expose lpf_get_rcvd_msg_count function. Also halfway (hopefully) through integrating BSC changes to enable both local and remote completion queues, which is key if we want to read t…
KADichev
committed
8b6b0f8
ibv_post_recv in new version fails at reconnectQPs
KADichev
committed
9e5becb
This version completes with HiCR, but still does not register ANY received events.
KADichev
committed
ea25ad7

Commits on Sep 28, 2023

Very importantly, remove sleeps in the progress engine, as this leads us to notice new reads/writes too late.
KADichev
committed
97cce7d

Commits on Sep 30, 2023

Enable functionality to associate a received message with its memory slot. This is currently done via imm_data field which carries the memory slot ID of the destination at the sender before it is R…
KADichev
committed
c5c86cd

Commits on Oct 1, 2023

Change IBVerbs::put to accept an original slot ID and the possibly modified slot ID if edge buffer is used. The original slot ID is then only used as a key for hashtable with key = slot ID and valu…
KADichev
committed
be6ecc2

Commits on Oct 4, 2023

Commits on Oct 13, 2023

Main changes here: 1) Implemented a round-robin put-based allgatherv within LPF as I need it. 2) Add get_rcvd_msg_cnt_per_slot besides the more general get_rcvd_msg_cnt, as the counts should be per…
KADichev
committed
cd4febc

Commits on Oct 17, 2023

Minor cleanup
KADichev
committed
102a35b

Commits on Oct 20, 2023

Commits on Oct 25, 2023

Commits on Nov 8, 2023

Add support for counting sent messages, and for tagged synchronization call with expected sent and expected received messages as parameters. The tagged synchronization call without expected sent an…
KADichev
committed
6753c74

Commits on Nov 15, 2023

Commits on Nov 23, 2023

Start work on compare and swap
KADichev
committed
64ef20b

Commits on Nov 26, 2023

The attributes retry_cnt and rnr_retry were set to 6 and 0 for development. Now set to 7 / 7 for infinite polling, if needed.
KADichev
committed
b9107bd

Commits on Nov 29, 2023

Make lookup of message counters pure lookup, no polling. This is tricky for HiCR, which then needs to do sync explicitly before checking these counters.
KADichev
committed
c860ced

Commits on Dec 15, 2023

Some very early documentation of the extensions in lpf/core.h, used in HiCR
KADichev
committed
1fd51b1

Commits on Jan 4, 2024

Minor improvements - use ibv_destroy explicitly in shared_ptr reset call in a few missing cases. Also remove tryLock/tryUnlock in this version, as it is not used yet.
KADichev
committed
b08d0e8

Commits on Jan 5, 2024

Remove debug output
KADichev
committed
5eea431

Commits on Jan 13, 2024

It seems to me that m_numMsgs was a wrong counter which included initiated sends and initiated receives. Now replace with a counter only for initiated sends. This counter is checked (initiated send…
KADichev
committed
ce28346

Commits on Feb 13, 2024

Add as backup semi-finished compare-and-swap example
KADichev
committed
b7917bc

Commits on Feb 26, 2024

Commits on Mar 1, 2024

Commits on Mar 5, 2024

Improvements for atomic compare-and-swap operation. Among them, now call wait_completion. Wait_completion is extended now to return the ibv_wc_opcode list, to check if events are atomic compare-and…
KADichev
committed
19554a5

Commits on Mar 11, 2024

Reorganize IBVerbs::get to register an Op::GET event. Sends are now basically either Op::SEND or Op::GET (put or get - both sends). Still lots of debug output
KADichev
committed
4b9529a

Commits on Mar 20, 2024

Commits on Mar 26, 2024

Commits on May 21, 2024

A modification replacing hash tables with arrays for all the counters, which significantly improves over ordered map implementation. Currently, it is fixed size 1000. This should be improved in cas…
KADichev
committed
5c3515d

Commits on Aug 2, 2024

WIP to merge hicr and main branch. Main goal: Have hicr as a new engine, instead of replacing existing engines
KADichev
committed
a2e4d75

Commits on Aug 13, 2024

This compiles, no idea if it works
KADichev
committed
048f4fa

Commits on Aug 14, 2024

Still working on getting LPF IB verbs tests to pass.
KADichev
committed
c502986
Towards working version
KADichev
committed
01b9e40

Commits on Aug 15, 2024

Minor alignment of ibverbs*, but a major fix in src/MPI/CMakeLists.txt to add macros for LPF_CORE_MPI_USES - without it, standalone ibverbs tests will compile incorrectly.
KADichev
committed
1f5b3a0

Commits on Aug 16, 2024

Minor
KADichev
committed
a62ca99