Trouble using external assemblies

[nwamsley1]

I have been having trouble using external assemblies. Following the readme I have tried the following:

using DotNET
reader = T"System.Reflection.Assembly".LoadFrom(raw"/Users/n.t.wamsley/Projects/Julia_Testing/ThermoFisher.CommonCore.RawFileReader.dll")

Which returns

System.Reflection.RuntimeAssembly("ThermoFisher.CommonCore.RawFileReader, Version=5.0.0.88, Culture=neutral, PublicKeyToken=1aef06afb5abd953")

How can I use methods and types from this library? For example in the RawFileReader library,
there is an attribute 'RawFileReaderAdapter' with a method 'FileFactory'. I would like to do something like

rawFile = RawFileReaderAdapter.FileFactory(filename)

rawFile then has methods I could call to return data from the file.

The problem is that if I enter the following

rawfilereaderadapter = reader.GetType("ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter", true, true)
filefactory = rawfilereaderadapter.GetMethod("FileFactory")
filefactory.ReturnType
filefactory.ReturnTypeCustomAttributes

I get

ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter
System.Reflection.RuntimeMethodInfo("ThermoFisher.CommonCore.Data.Interfaces.IRawDataExtended FileFactory(System.String)")
ThermoFisher.CommonCore.Data.Interfaces.IRawDataExtended
System.Reflection.RuntimeParameterInfo("ThermoFisher.CommonCore.Data.Interfaces.IRawDataExtended")

respectively. Now I need to invoke the FileFactory method.

julia> filepath = convert(CLRObject, "/Users/n.t.wamsley/Projects/SAGE_TESTING/MA4358_FFPE_HPVpos_01_071522.raw")
julia> rawfile = filefactory.Invoke("", [filepath])
ThermoFisher.CommonCore.RawFileReader.RawFileAccess("ThermoFisher.CommonCore.RawFileReader.RawFileAccess")
julia> rawfile.IsOpen
false

rawfile.IsOpen should be returning true. I have verified that the file path is correct. So I suspect I am passing methods incorrectly. I wasn't sure if you had any ideas about what I could be doing wrong?

Here is a snippet from the .xml file describing the types and methods defined in the dll

 <member name="T:ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter">
            <summary>
            This static class contains factories to open raw files
            </summary>
        </member>
        <member name="M:ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter.FileFactory(System.String)">
            <summary>
            Create an IRawDataExtended interface to read data from a raw file
            </summary>
            <param name="fileName">File to open</param>
            <returns>Interface to read data from file</returns>
        </member>

[nwamsley1]

There is a clearer explanation of the problem with a minimal example and context on stack overflow. https://stackoverflow.com/questions/75325784/using-a-net-assembly-in-with-dotnet-jl-julia

[azurefx]

Hi nwamsley1,

You can invoke instance methods or static methods directly without explicit use of reflection. If the invocation raises an exception, it will be thrown in Julia as a CLRException.

julia> using DotNET

julia> T"System.Reflection.Assembly".LoadFrom("ThermoFisher.CommonCore.RawFileReader.dll")
System.Reflection.RuntimeAssembly("ThermoFisher.CommonCore.RawFileReader, Version=5.0.0.88, Culture=neutral, PublicKeyToken=1aef06afb5abd953")

julia> RawFileAdapter = T"ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter, ThermoFisher.CommonCore.RawFileReader"
ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter

julia> rawFile = RawFileAdapter.FileFactory("empty")
ThermoFisher.CommonCore.RawFileReader.RawFileAccess("ThermoFisher.CommonCore.RawFileReader.RawFileAccess")

julia> rawFile.IsOpen
false

julia> rawFile.IsError
true

There is no exception, indicating that the RawFileAccess object has been successfully created.

I'm not familiar with this library, and I don't have sample data that can be read by this FileFactory. You can check the documentation of this library to see if there is an error code or other way to check for errors.

[azurefx]

I checked the DLL and found there is actually an exception:

julia> fileError = rawFile.FileError
ThermoFisher.CommonCore.RawFileReader.Facade.RawFileLoader("ThermoFisher.CommonCore.RawFileReader.Facade.RawFileLoader")

julia> fileError.ErrorCode
-1

julia> fileError.WarningMessage
"Information: Creating mutex for: empty.raw\r\n"

julia> fileError.ErrorMessage
"Encountered problems while trying to read '' as a Raw File!\r\n\r\nException type: System.MissingMethodException\r\nMessage       : Method not found: 'Void System.Threading.Mutex..ctor(Boolean, System.String, Boolean ByRef, System.Security.AccessControl.MutexSecurity)'.\r\nStacktrace:\r\n   at ThermoFisher.CommonCore.RawFileReader.Utilities.CreateNamedMutexAndWait(String fileName, Boolean useGlobalNamespace, DeviceErrors errors)\r\n   at ThermoFisher.CommonCore.RawFileReader.Facade.RawFileLoader..ctor(String fileName)\r\n"

The reason is that RawFileReader calls the Mutex constructor which has a MutexSecurity parameter that is only supported by .NET Framework, indicating that this library may only run on .NET Framework.

DotNET currently only supports cross-platform .NET (Core) environment (#19) but I may add .NET Framework 4.X support on Windows in the future.

[nwamsley1]

Thanks, this is very helpful. It turns out that Thermo has a .NET standard 2.0 build of these libraries (I think this would work?). I requested it, so I think I will be able to get this to work after all. I can replicate what you are showing on my machine. Thanks for you help!

[nwamsley1]

With the .NET Core 2.0 build, I encounter an error where a space seems to be added to the front of the filepath.

julia> filepath = convert(CLRObject,"/Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw")
julia> rawFile = RawFileAdapter.FileFactory(filepath)
julia> fileError = rawFile.FileError
julia> fileError.ErrorMessage
" /Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw"
julia> fileError.WarningMessage
"Information: Creating mutex for: /Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw\nInformation: Created mutex for: /Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw\nInformation: Release mutex for: /Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw\nInformation: Close mutex for: /Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw\n"

[azurefx]

This error message doesn't seem informative enough.. Are there any error codes?

By the way, there should be no spaces at the beginning of the string. This can be easily verified:

julia> juliastr = "/Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw"
"/Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw"

julia> length(juliastr)
74

julia> clrstr = convert(CLRObject, juliastr)
System.String("/Users/n.t.wamsley/Projects/Julia_Testing/MA4358_FFPE_HPVpos_01_071522.raw")

julia> clrstr.Length
74

# no need to `convert` a Julia string to `CLRObject` since most primitive types in Julia are automatically converted back and forth
julia> clrstr.StartsWith("/Users")
true

I suspect that the space is caused by string interpolation, and the leading error message somehow disappeared. If you can load the .NET Core library in Python, can you try to see if you get the same error?

[nwamsley1]

The error code is 2. I verified the string length, just strange that the error message seems to be adding a space in what it prints out.
From the xml

 <member name="P:ThermoFisher.CommonCore.RawFileReader.Writers.DeviceErrors.ErrorCode">
            <summary>
            Gets the error code number.
            Typically this is a windows system error number.
            If no number is encoded:
            Returns "1" if the error message is null
            Returns "2" is there is an error message.
            A return of "0" will occur if there has been no error message
            </summary>
        </member>

From some googling, it seems like the windows error code 2 has to do with a "can't find the file specified" type error.

[azurefx]

0x2 means ERROR_FILE_NOT_FOUND: The system cannot find the file specified.

[azurefx]

I don't know how to debug system calls under *nix, but if you are running Windows, you can try the procmon tool from Sysinternals tookit, which records disk activities of a specific process. Then you can check for failed CreateFile(A|W) or other Win32 API calls related to the file system.

Can you share the .NET Core version of the library and a valid raw file? I can help you debug it on a Windows computer.

[nwamsley1]

That would be awesome.

https://figshare.com/articles/dataset/Raw_and_DLL/22022552

I think this link will work.

[azurefx]

It works fine on my computer.

julia> using DotNET

julia> T"System.Reflection.Assembly".LoadFrom("ThermoFisher.CommonCore.RawFileReader.dll")
System.Reflection.RuntimeAssembly("ThermoFisher.CommonCore.RawFileReader, Version=5.0.0.88, Culture=neutral, PublicKeyToken=1aef06afb5abd953")

julia> RawFileAdapter = T"ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter, ThermoFisher.CommonCore.RawFileReader"
ThermoFisher.CommonCore.RawFileReader.RawFileReaderAdapter

julia> rawFile = RawFileAdapter.FileFactory("MA4358_FFPE_HPVpos_01_071522.raw")
ThermoFisher.CommonCore.RawFileReader.RawFileAccess("ThermoFisher.CommonCore.RawFileReader.RawFileAccess")

julia> rawFile.IsOpen
true

julia> rawFile.IsError
false

julia> rawFile.ComputerName
"THERMO-2KV4HQ2"

julia> rawFile.InstrumentCount
4

julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: AMD Ryzen 7 3700X 8-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, znver2)
Environment:
  JULIA_NUM_THREADS = 4

julia> DotNET.detect_runtime(DotNET.CoreCLR.CoreCLRHost)
6-element Vector{Any}:
 (type = "Microsoft.NETCore.App", version = v"6.0.13", path = "C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\6.0.13\\coreclr.dll")
 (type = "Microsoft.NETCore.App", version = v"3.1.32", path = "C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\3.1.32\\coreclr.dll")
 (type = "Microsoft.NETCore.App", version = v"3.0.1", path = "C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\3.0.1\\coreclr.dll")
 (type = "Microsoft.NETCore.App", version = v"2.0.9", path = "C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\2.0.9\\coreclr.dll")
 (type = "Microsoft.NETCore.App", version = v"2.0.7", path = "C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\2.0.7\\coreclr.dll")
 (type = "Microsoft.NETCore.App", version = v"2.0.6", path = "C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\2.0.6\\coreclr.dll")

DotNET will choose the first runtime returned by detect_runtime. What version of .NET are you using?

[nwamsley1]

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin21.4.0)
  CPU: 16 × Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 16 virtual cores

julia> DotNET.detect_runtime(DotNET.CoreCLR.CoreCLRHost)
9-element Vector{Any}:
 (type = "Microsoft.NETCore.App", version = v"7.0.2", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.2/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"5.0.17", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/5.0.17/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"5.0.13", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/5.0.13/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"5.0.12", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/5.0.12/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"5.0.11", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/5.0.11/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"3.1.26", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.1.26/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"3.1.22", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.1.22/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"3.1.21", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.1.21/libcoreclr.dylib")
 (type = "Microsoft.NETCore.App", version = v"3.1.20", path = "/usr/local/share/dotnet/shared/Microsoft.NETCore.App/3.1.20/libcoreclr.dylib")

I am running a different version than you are. I tried removing 7.0.2 so that it would use the earlier version. That did not work. So this could be something MacOS specific.

[nwamsley1]

From Thermo's documentation

// This code has been compiled and tested using Visual Studio 2013 Update 5, Visual Studio 
// 2015 Update 3, Visual Studio 2017 Update 2, Visual Studio 2019, and Visual Studio 2022, Visual
// Studio MAC, and MonoDevelop.  Additional tools used include Resharper 2017.1.  This application
// has been tested with .Net Framework 4.5.1, 4.5.2, 4.6, 4.6.1, 4.6.2, 4.7, and 4.8.

[nwamsley1]

I think it is strange that in all the file error messages I get, a space is appended to the front of the file paths. Even though the string I provide clearly does not have the space. Could be a red herring, but if not I could try and figure out at what step that space gets added.