SushiHangover

PowerShell, Learn it or Perish ;-)

master nix
Gitter

MS CorFlags - Showing incorrect flags

In the Microsoft supplied CorFlags.exe, when displaying the flags on a COMIMAGE_FLAGS_32BITPREFERRED && COMIMAGE_FLAGS_32BITREQUIRED flagged CIL PE image, it does not show that the COMIMAGE_FLAGS_32BITREQUIRED as being set.

In the case of this output:

Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 0x20003
ILONLY    : 1
32BITREQ  : 0
32BITPREF : 1
Signed    : 0

The 0x20003 flags breaks down into:

  • 32BITPREFERRED (0x20000)
  • 32BITREQUIRED (0x00002)
  • ILONLY (0x00001)

So why is 32BITREQ not shown as selected?

My xplat version of CorFlags shows the following:

Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 0x20003
ILONLY    : 1
32BITREQ  : 1
32BITPREF : 1
Signed    : 0

The 32BITREQ is shown as flagged as the value of 0x20003 includes that flag…. so who is right?

Microsoft in hiding that the 32BITREQ flag is set on PEs that have 32BITPREF set (while still showing the actual flag value), is that right? a bug? Should I do the same in my xplat version of the tool? Let me know in the comments below, or even better post an issue over at my xplat CorFlags project (Thanks).

Misc:

In setting the compiler /platform flag to ‘anycpu32bitpreferred’ you are targeting 32-bit even if on a 64-bit system, thus the 32BITREQ and 32BITPREF flags are set.

anycpu32bitpreferred compiles your assembly to run on any platform. Your application runs in 32-bit mode on systems that support both 64-bit and 32-bit applications. You can specify this option only for projects that target the .NET Framework 4.5. (Or Mono 4.5...)

IMAGE_COR20_HEADER.Flags values from \Include\um\CorHdr.h

COMIMAGE_FLAGS_ILONLY               =0x00000001,
COMIMAGE_FLAGS_32BITREQUIRED        =0x00000002,
COMIMAGE_FLAGS_IL_LIBRARY           =0x00000004,
COMIMAGE_FLAGS_STRONGNAMESIGNED     =0x00000008,
COMIMAGE_FLAGS_NATIVE_ENTRYPOINT    =0x00000010,
COMIMAGE_FLAGS_TRACKDEBUGDATA       =0x00010000,
COMIMAGE_FLAGS_32BITPREFERRED       =0x00020000,

PlayScript - Moved it a Github organization

I transfered my PlayScript repo to an Github organization to help maintain the structure of the project.

The new org is named PlayScriptRedux and any forks should be made from there as it will be the master and now my personal PlayScript repo will also be a fork of that master.

I’ll be (slowly) posting the other PlayScript related repo (Starling, Away3d, IDE add-ins, etc.) to the PlayScriptRedux as they are tested.

What is Code

No matter what the Slate says, Paul Ford’s “What is Code” desires to be read, if not by every programmer, then surely by every person that has to deal with programmers.

The best thing, if you find any issues with the article, you can post a Github issue, I did. ;-)

Mono - Another benchmark between 32-bit and 64-bit Mono / LLVM

While I was still in the []benchmark mood](http://sushihangover.github.io/mono-64-bit-vs-32-bit-benchmark-using-scimark/), I thought I would run a perf test that someone wrote years ago. Not sure where it came from originally, but it includes 5 simple tests with only one being numeric related.

Per the Mono LLVM web page, LLVM is not able to support some of the features that Mono needs, so in those cases the JIT compiler will still fall back to Mono’s JIT engine (i.e. Methods that contain try/catch clauses or methods that do interface calls).

One of the included tests is throwing and catching 100,000 exceptions and the 32/64-bit results show that enabling LLVM does not help.

    Note: The Mono web pages are notorious for begin out of date or lacking of details, even the 'new' ones, so I usually like confirmation. 

As the SciMark test showed, numeric operations under LLVM native code jit totally rocks over the built-in one, over 2x faster in this simple test.

String related functions can be 2x faster in the 64-bit mono, but enabling LLVM for either 32 or 64-bit does not really help that much, LLVM is still better but by micro-seconds (uS).

Enabling LLVM in Mono:

  • Methods with Exceptions: No Help at all
  • Numerics: Still rule
  • String functions: No much help using LLVM, but 64-bit Mono kicks its 32-bit brother in the butt.
  • Refection: Not much help either, as expected, but again 64-bit rules.

Net Results:

I am still not seeing a reason not to use a 64-bit version of Mono and not enabling LLVM as I am not using any 32-bit frameworks on OS-X (i.e.: OS-X Carbon API wrappers or GTK# 2.x).

Try it yourself: Gist

mcs -optimize+ -debug- -sdk:4.5 -target:exe -platform:x64 perftest.cs -out:perftest-64-release.exe

Next up: How much does AOT'ing your CIL exe/dll images help…or not.)

Additional Reading:

Sample output:

$which mono
/Users/administrator/mono/mono-llvm-64/bin/mono

$mcs -optimize+ -debug- -sdk:4.5 -target:exe -platform:x64 perftest.cs -out:perftest-64-release.exe

$mono perftest-64-release.exe 
ArrayList strings test.............408 ms
StringBuilder test.................499 ms
Integer & Floating ADD.............3616 ms
Exception test.....................375 ms
Reflection and recursion...........2730 ms

$ mono --llvm perftest-64-release.exe 
ArrayList strings test.............497 ms
StringBuilder test.................505 ms
Integer & Floating ADD.............1620 ms
Exception test.....................407 ms
Reflection and recursion...........2790 ms

$which mono
/Users/administrator/mono/mono-llvm-32/bin/mono

$mcs -optimize+ -debug- -sdk:4.5 -target:exe -platform:x86 perftest.cs -out:perftest-32-release.exe

$mono perftest-32-release.exe 
ArrayList strings test.............777 ms
StringBuilder test.................958 ms
Integer & Floating ADD.............3610 ms
Exception test.....................589 ms
Reflection and recursion...........4557 ms

mono --llvm perftest-32-release.exe 
ArrayList strings test.............830 ms
StringBuilder test.................873 ms
Integer & Floating ADD.............1731 ms
Exception test.....................617 ms
Reflection and recursion...........4441 ms

Mono - 64-bit vs. 32-bit benchmark using SciMark

I wanted to compare 64-bit and 32-bit mono versions and PE32 vs PE32+ CIL images and see if 64-bit really is faster, at least on the number crunching side of things.

This is the C# port of the Scimark benchmark. The original C and Java sources can be found at http://math.nist.gov/scimark2/. The original work to port the benchmark to C# was done by Chris Re and Wener Vogels of the Rotor project.

https://code.google.com/p/scimark-csharp

Running an Apples to Apples comparision across platforms and different mono builds using different llvm versions jitting is not really a valid test-case. So the following benchmark matrix is based on the same clang/llvm version compiling the same mono version in x64 (PE32+) and i386 (PE32) archs on just one platform (my ‘old’ ‘MacBookPro).

Compile four different CIL (exe) images:

  • x64 and x86 Mono Debug (Debug Full, no optimize)
  • x64 and x86 Mono Release (No Debug, Optimized)

Execute 32-bit Mono and 62-bit Mono versions loading the respective CIL (exe) images:

  • LLVM turned off
  • LLVM turned on (–llvm)

The results are not really that surprising in regards to is 64-bit number crunching faster than 32-bit, yes, 64-bit it faster, about 1.2x faster. The surprise for me is the LLVM results, both in x86 and x64 versions of mono/llvm builds, mono enabled LLVM produces code that is 1.5x faster under SciMark. This speed difference is well worth the slightly longer startup times that jitting with LLVM causes.

Get the SciMark repo on my GitHub and run it on your platform.

CPU Arch Build Type LLVM Score (MFlops) Notes:
x86 Debug 371.42
x86 Debug Yes 465.45 1.2 times faster using LLVM
x86 Release 295.11
x86 Release Yes 448.92 1.5 times faster using LLVM over not
using it in Release mode
x64 Debug 367.22
x64 Debug Yes 559.28
x64 Release 370.79
x64 Release Yes 569.46 1.2 times faster than x86 using LLVM
1.5 times faster than x64 without LLVM

Mono - libtool: Version mismatch error. This is libtool 2.4.6

If you are receiving the following error when compiling mono 4.x “Version mismatch error. This is libtool 2.4.6, but the definition of this LT_INIT comes from libtool 2.4.2.”, then the aclocal.m4 file needs to be rebuilt. It appears something in the mono autogen/configure and/or make clean is not deleting and rebuilding it (or is picking an older libtool vs. actually using glibtool).

I have find that the “libgc/aclocal.m4” has been my main problem:

 ....   
    make[4]: Nothing to be done for `all-am'.
    Making all in doc
    make[3]: Nothing to be done for `all'.
      CC       pthread_support.lo
    libtool: Version mismatch error.  This is libtool 2.4.6, but the
    libtool: definition of this LT_INIT comes from libtool 2.4.2.
    libtool: You should recreate aclocal.m4 with macros from libtool 2.4.6
    libtool: and run autoconf again.
    make[3]: *** [pthread_support.lo] Error 63
    make[2]: *** [all-recursive] Error 1
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2

Remove that file:

rm libgc/aclocal.m4 

And re-run the make and it will be recreated correctly.

Mono - Speeding up your llvm compile

If you are building llvm so you can then configure and build mono to use it, then you can speed up the compile by removing the tests as they take as long, if not longer, then the llvm runtime binaries.

I could not find a llvm configure option to skip the test directory, but you you just delete it (or move it out of the llvm directory) before you configure/compile llvm, then you will receive a warning that it was not find, but the compile will proceed normally.

Note: You could also mod the llvm configuration (configure/Makefile) to skip the tests.

Mono - Using LLVM? Make sure your targets match

When configuring Mono to use llvm, if you receive the error:

configure: error: Cross compiling is not supported for target i386-apple-darwin11.2.0

Then the llvm-config that is currently the first hit in your path and that is begin picked up by the mono autogen/configure script is does not match the target that you are trying to build in mono.

llvm-config –host-target should help you out and change your path so the correct llvm bin directory is first in the list so when mono’s configure runs llvm-config it gets the correct info.

Stripping the build prefix from cross-compiled binaries

Remove a filename prefix from a directory of files

When cross-compiling and using the standard autogen/configure “-target=” option, you end up with binaries that have the arch prefix on your binaries. While this makes sense in order to distinguish that the binaries are different, there are times you do not want to deal with the issue of using these prefixes all the time.

Here is a simple script to strip those prefixes:

#!/bin/bash
if [ -z "$1" ]
then
  prefix="i386-apple-darwin14.3.0-"
else
  prefix=$1
fi
for filename in `ls $prefix*`
do
   newfilename=${filename:${#prefix}}
   echo $filename $newfilename
   mv $filename $newfilename
done

I seem to be stripping “i386-apple-darwin14.3.0-” lately so that is the default if no prefix is passed as an argument…

OS-X 64-bit Mono : Midnight Commander

In testing out 64-bit ncurses on OS-X, I grabbed the C# version of Midnight Commander to do some manual testing.

After tweaking the Makefile for PE32+ images and changing my path to include a 64-bit mono, I ended up with the needed project’s exe, dll, and dylib:

file *.exe *.dll *.dylib
mc.exe:               PE32+ executable for MS Windows (console) Mono/.Net assembly
mono-curses.dll:      PE32+ executable for MS Windows (DLL) (console) Mono/.Net/assembly
libmono-curses.dylib: Mach-O 64-bit dynamically linked shared library x86_64

There are some minor painting issues which I do not know if they are related to the arch type, OS-X terminal or bugs in the actual C# code (mc or the ncurses wrapper), but overall it works very well.