Maybe we should relax inplace optimization conditions in plan_memory.cc?

[shuo-ouyang]

Hi guys, we developed our custom fake quantization operator but suffer out of memory issues due to store activations two times (before and after fakequant op). We try to solve this problem by making forward and backward in place, however, the in_data and out_data are not shared because the mxnet backend makes the final decision of memory optimization. We notice that the condition at line 257 in plan_memory.cc controls whether to perform optimization, in our opinion, storage_ref_count[sid_in] will be large than 1 in some cases (at least in our models). We relax the condition to storage_ref_count[sid_in]>=1 and solve the OOM issue. Maybe we can relax this condition and expand to other models?

https://github.com/apache/incubator-mxnet/blob/5d247f13fcf5e55b094c5deb90ede1d3a03cc9ac/src/nnvm/plan_memory.cc#L257

[shuo-ouyang]

I checked the code and noticed that the FInplaceIdentity function enables in-place optimization even when the input reference count is greater than one, which covers the above cases.