Too slow inference time

[psh117]

Neural ODE requires multiple inference of the networks (60~100 times) but each inference takes about 3 ms, which means that it took 200~300 ms for an IK solution.

However, sampling 1,000,000 configuration took only 2.87 s. (because of the batch calculation; using GPU Tesla v100)
How do you think about this issue?
Maybe I can reduce the "inference time" of the network by minimzing the network structure.

btw I uploaded sample codes to reproduce the results and the time measurement.

[cadop]

I think we can reduce the network to only what is needed, perhaps using the grid functionality to sample the hyperparameter space.

What is the inference you are using? I don't think you should be rerunning the inference multiple times, its a continous space, so depth of the network should be based on the sampling tolerance

[cadop]

okay I see in the notebook it was number of function evals that you were describing. How does it scale though? Aren't you using max_len at ~50,000 or something? So its more than one IK solution?

[psh117]

Yes right. I just updated the example.ipynb. Sampling time for a single solution is almost same with 512. I'll try with some optimized code.

Do you think if it is worthwhile to generate many IK solutions? If then, it can provide fast IK solutions for batch request.

[psh117]

I'm using the FC network with 3 layers of 64 hidden dimension

joint q, x (affine coupling) -- in (7) -- 64 -- 64 -- 64 -- out (7)

[cadop]

the resolution of network seems reasonable to me.

I think the key is that single solution is almost same as 512. so its more appropriate to try and sample 1,000,000 solutions to see how long it takes.

the other consideration is that we are doing the trivial case now, that has a well known/optimized analytic solution. But the hope is that when adding multiple dof's and multiple robots, the complexity is high enough that things like mesh collisions are too expensive for the numeric IK solution