It's tricky to come up with a succinct title to describe the quirk that is the topic of today's post! Hopefully it contains the right keywords for someone who comes across a similar issue to find it.
The CI builds of the C#/.Net Elasticsearch client publish versioned packages of the client and dependency packages, to allow benchmarking different client versions against one another, in addition to helping users in upgrading their application and Elasticsearch to the next major version. Versioned packages are produced by rewriting assemblies with Mono.Cecil, functionality that is conveniently wrapped up in a dotnet CLI tool, AssemblyRewriter. In essence, the process of rewriting an assembly involves rewriting all namespaces therein and renaming it, and doing the same for its direct dependencies (excluding framework assemblies), resulting in a differently named assembly with the same functionality.
Take the Elasticsearch.Net assembly built from the master branch as an example. The master branch is currently targeting the 8.0.0-SNAPSHOT version of Elasticsearch. Rewriting the assembly to a versioned one produces an assembly named Elasticsearch.Net8, and the client class is rewritten from
namespace Elasticsearch.Net
{
public partial class ElasticLowLevelClient : IElasticLowLevelClient
{
// ...
}
}
to
namespace Elasticsearch.Net8
{
public partial class ElasticLowLevelClient : IElasticLowLevelClient
{
// ...
}
}
This means that Elasticsearch.Net8 package and Elasticsearch.Net package from Nuget (which might be the latest 7.x version) can both be referenced in the same project.
Dynamic assemblies, strong naming, and friend assemblies
One of the challenges in rewriting the client assemblies is that the JSON serializer used by the 7.x version of the client, a fork of Utf8Json, heavily uses Reflection.Emit to generate IL at runtime, creating dynamic assemblies. Since the client assemblies are strongly named, and dynamic assemblies require access to internal types within the client assemblies, dynamic assemblies are also strongly named, and the new .NET SDK project files of the client use the InternalsVisibleTo item elements to make internal types visible to dynamic assemblies.
For example, Elasticsearch.Net's project file looks like
<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<PackageId>Elasticsearch.Net</PackageId>
<Title>Elasticsearch.Net</Title>
<PackageTags>elasticsearch;elastic;search;lucene;nest</PackageTags>
<Description>
Exposes all the Elasticsearch API endpoints but leaves you in control of building the request and response bodies.
Comes with built in cluster failover/connection pooling support.
</Description>
</PropertyGroup>
<ItemGroup>
<InternalsVisibleTo Include="Nest" />
<InternalsVisibleTo Include="Tests" />
<!-- Dynamic assemblies -->
<InternalsVisibleTo Include="Elasticsearch.Net.CustomDynamicObjectResolver" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicCompositeResolver" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateOriginal" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateCamelCase" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateSnakeCase" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateOriginal" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateCamelCase" />
<InternalsVisibleTo Include="Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateSnakeCase" />
</ItemGroup>
<!-- ...etc. -->
</Project>
An MSBuild target in a Directory.Build.targets file in the root of the solution takes the <InternalsVisibleTo> elements as input, and emits System.Runtime.CompilerServices.InternalsVisibleTo assembly attributes with public keys for both versioned and non-versioned assemblies
<Target Name="AssemblyInfos"
BeforeTargets="CoreGenerateAssemblyInfo"
Inputs="@InternalsVisibleTo" Outputs="%(InternalsVisibleTo.Identity)"
Condition="$(IsPackable) == True"
>
<PropertyGroup>
<ExposedAssembly>%(InternalsVisibleTo.Identity)</ExposedAssembly>
<VersionNamespaced>$(ExposedAssembly.Replace("Nest", "Nest$(MajorVersion)").Replace("Elasticsearch.Net","Elasticsearch.Net$(MajorVersion)"))</VersionNamespaced>
</PropertyGroup>
<ItemGroup>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo">
<_Parameter1>%(InternalsVisibleTo.Identity), PublicKey=$(ExposedPublicKey)</_Parameter1>
</AssemblyAttribute>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo">
<_Parameter1>$(VersionNamespaced), PublicKey=$(ExposedPublicKey)</_Parameter1>
</AssemblyAttribute>
</ItemGroup>
</Target>
When compiling from an IDE like JetBrains Rider, the Elasticsearch.Net.AssemblyInfo.cs file in obj/$(Configuration)/$(TargetFramework)/ generated when building the project ends up looking similar to
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Nest, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Nest8, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.CustomDynamicObjectResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.CustomDynamicObjectResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicCompositeResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicCompositeResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net8.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Tests, PublicKey=<PublicKey here>")]
The problem
A user opened a detailed issue indicating they were getting TypeLoadException: Access Denied exceptions when using a 7.x versioned assembly from our client CI feed. An investigation confirmed the finding, with the following AssemblyInfo written when building from the command line with dotnet build
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Nest, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Nest8, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.CustomDynamicObjectResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.CustomDynamicObjectResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicCompositeResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicCompositeResolver, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullFalseNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateOriginal, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateCamelCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Elasticsearch.Net.DynamicObjectResolverAllowPrivateFalseExcludeNullTrueNameMutateSnakeCase, PublicKey=<PublicKey here>")]
[assembly: System.Runtime.CompilerServices.InternalsVisibleTo(@"Tests, PublicKey=<PublicKey here>")]
Why was only the first InternalsVisibleTo assembly attribute for Nest versioned, but none of the rest? And why were they repeated?
Investigating with MSBuild Binary and Structured Log Viewer
Starting with MSBuild 15.3 (Visual Studio 2017 Update 3 or newer), MSBuild supports serializing all build events to a binary log file, with the /bl switch. What's great about this is that, combined with the excellent MSBuild Binary and Structured Log Viewer, it is easy to visualize and investigate builds. The switch can be passed to the dotnet CLI tool with
dotnet build project.csproj /bl
outputting an msbuild.binlog that can be opened with the log viewer.
Running this on the command line build of Elasticsearch.Net showed that the build target was running as expected
Looking at the generated Elasticsearch.Net.AssemblyInfo.cs file in obj/$(Configuration)/$(TargetFramework)/ however still showed that the InternalsVisibleTo assembly attributes for Elasticsearch.Net.* were not versioned
Why were versioned InternalsVisibleTo assembly attributes written correctly when building from an IDE, but not from the command line?
Back to the docs
The documentation for friend assemblies notes the following
When you compile an assembly like AssemblyB that will access internal types or internal members of another assembly like Assembly A, you must explicitly specify the name of the output file (.exe or .dll) by using the -out compiler option. This is required because the compiler has not yet generated the name for the assembly it is building at the time it is binding to external references.
The build from the commandline does not pass an output path, relying on the convention of output going to the /bin/$(Configuration)/$(TargetFramework)/ directory of the project. When the IDE builds projects however, an output path is explicitly passed. Could this be the issue?
Passing an explicit output path as part of the command line build yielded the expected result; versioned InternalsVisibleTo assembly attributes were correctly written to the Elasticsearch.Net.AssemblyInfo.cs, fixing the reported issue.
Since all client projects are built from the commandline by building the solution, the fix was to add
<PropertyGroup>
<OutDir>bin/$(Configuration)/$(TargetFramework)/</OutDir>
</PropertyGroup>
to the Directory.Build.targets file in the root of the solution.
Summary
I guess the TLDR; of this post is to read the documentation :) More seriously though, I thought this was an interesting enough quirk to write about. Hopefully, someone more familiar with build internals and InternalsVisibleTo semantics might be able to comment on why the Nest versioned attribute was written, but not the Elasticsearch.Net.* attributes. I'd be interested to better understand how this works.