Currently, example 301 which evaluates pre-trained CNTK model with CIFAR10 images, is totally written by python. This example use pickle.load to read cifar-10-batches-py/test_batch and then parallelize to distributed RDD, however, I cannot directly use pickle to read data in scala code application.
I tried spark.readImages in mmlspark, but it seems cannot deal with cifar-10-batches-bin data well. And I finally choose cookie-datasets to read cifar data in scala (the master branch of cookie-datasets still used spark-1.5, and I upgrade it to spark-2.1 with necessary changes)
BTW, since you only have python examples, and I have already interpreted 101 and part 301 examples code to scala, I'm not sure whether you want this part of example codes?
Currently, example 301 which evaluates pre-trained CNTK model with CIFAR10 images, is totally written by python. This example use
pickle.loadto readcifar-10-batches-py/test_batchand then parallelize to distributed RDD, however, I cannot directly usepickleto read data in scala code application.I tried
spark.readImagesin mmlspark, but it seems cannot deal withcifar-10-batches-bindata well. And I finally choose cookie-datasets to read cifar data in scala (the master branch of cookie-datasets still used spark-1.5, and I upgrade it to spark-2.1 with necessary changes)BTW, since you only have python examples, and I have already interpreted 101 and part 301 examples code to scala, I'm not sure whether you want this part of example codes?