x
This website is using cookies. We use cookies to ensure that we give you the best experience on our website. More info. That's Fine
HPC:Factor Logo 
 
Latest Forum Activity

Another development environment option!

1 2 3
smb_gaiden Page Icon Posted 2018-04-01 2:57 AM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Well, came across another compiler and observed they added ARM support about 4-5 years ago. Then there was an update in December 2017 fixing some stuff and adding ARM64 support. The former was intriguing and the latter was not applicable.

Introducing Tiny C compiler for Windows CE HPC development! The author is here: https://bellard.org/tcc/ . The download is here: http://download.savannah.gnu.org/releases/tinycc/ . It needs the .tar.bz2 package, so it includes sources.

tldr; used tcc to compile an executable that ran successfully on the jornada 820.

My adventure follows:

1. Downloaded the .tar.bz2
2. unpacked it on linux (chromeos using crouton to a trusty chroot)
3. Compiled it to test the general native compiler
4. built a helloworld native app to test, and observed success
5. built the compiler again for all cross platforms
6. wrote a simple Windows CE app to show a MessageBoxW
MessageBoxW(0, "T\0e\0s\0t\0\0", "T\0e\0s\0t\0\0", 0);
7. needed to make a COREDLL.def to link with for the MessageBoxW symbol
8. compiled with the arm-wince-tcc
9. Ran it on Jornada 820 and it failed with some unloaded dependencies
10. Used embedded visual tools 3.0 to compile a similar app without any external dependencies such as wWinMainCRT or whatever it is called.
11. Observed the binary generated by EVT3 ran on the jornada 820.
12. Did some work to extract the raw code from the .text section of the PE files of each respectively.
13. Used objdump to show the associated assembler code and read through it.
14. Determined both generated valid code, but differently.
15. Used a PE inspector to look at the differences between the TCC generated binary and the EVT3 generated binary.
16. Observed some differences within some of the optional headers fields
17. hex edited the TCC generated binary to match the EVT3 generated binary
18. Loaded the hex edited binary to the jornada 820 and ran it.


success!

I left out a bunch of steps of failure trying to use the wce211 sdk include files, fixing case sensitive header file names, and circular dependencies. That's for later.

What does it mean?
1. There's a possibility that the tcc project itself could be ported onto arm architecture in order to use jornada as a development machine in order to write software that runs on itself. First thing is first, which will be to edit tcc in order to accept command line options to control the areas I had to manually hex edit.
2. I don't have to boot into a virtual machine in order to write software targeting a Jornada 820 (and likely other ARM based CE devices).
3. Sanitizing the wce header files may make compile time checking easier to develop in this environment, blindly linking against dynamic dlls won't guarantee the caller is using the appropriate parameter count and types.

Addendum:

Here's what the compile command looked like:
./arm-wince-tcc -o MsgBawx.exe -nostdlib -DUNICODE -D_UNICODE -D_ARM_ -L . -lcoredll MsgBawx.c

Here's what the .def file looked like:
LIBRARY coredll.dll

EXPORTS
MessageBoxW

Here's the project source code:

void _start(void)
{
MessageBoxW(0, "T\0e\0s\0t\0\0", "T\0e\0s\0t\0\0", 0);
}

The MSVC project to compare against was whittled down to resemble the tcc project and removed or altered:
- cpp, and flattened to c
- resource files
- stdafx
- *.rc
- resource headers
- wWinMainCRT or whatever, just did the int SYMBOL(){} as an entry point (SYMBOL was some name that I used in the project settings as entry point)
- did not use command line arguments or hwnd arguments, just an empty paramter function returning int.
- generated own library, so they wouldnt link the dynamically linked library by ordinal
--- this involved making a def file as above
--- using lib.exe such as:
----- lib.exe /def:coredll.def /machine:arm /out:coredll.lib
---------- the above may have had ARM instead of lowercase arm


As a result the file sizes were:
TCC MsgBawx: 1,536
EVT3 MsgBawx: 3,072

The PE file comparison:
TCC MsgBawx has two sections (.data and .text)
EVT3 MsgBawx has four sections (.data, .text, .rdata, .pdata)

TCC MsgBawx has a smaller MS-DOS stub than the EVT3 MsgBawx.
 Top of the page
smb_gaiden Page Icon Posted 2018-04-04 8:49 PM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Copy stingraze and add a screenshot!





(msgbawx.png)



Attachments
----------------
Attachments msgbawx.png (11KB - 0 downloads)
 Top of the page
stingraze Page Icon Posted 2018-04-05 2:30 AM
#
Avatar image of stingraze
H/PC Vanguard

Posts:
3,656
Location:
Japan
Status:
This is very cool.

Do you mean the TCC can compile on Jornada itself to create binaries for itself?

-stingraze
 Top of the page
smb_gaiden Page Icon Posted 2018-04-05 4:32 AM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Ohaiyou!

Theoretically still - haven't heard of any projects which ported tcc to the wince device itself yet. Making it reality is my future goal actually!

Thoughts:
tcc compiles itself. For example, I compiled tcc and the tcc cross compilers (including arm-wince-tcc from source on ubuntu).
Then arm-wince-tcc compiles an arm binary that needs 3 bytes hex edited to successfully execute on the jornada. (May only need two, didn't test after success with three altered bytes), as long as in the hex editor 3 vs. 2 is insignificant. Next goal is to alter tcc src in order to add linker options to alter those bytes directly.
Therefore, if i use arm-wince-tcc to compile arm-wince-tcc and resolve std c library deficiencies it should run on the Jornada. Then it will be an exercise of debugging in case some RAM/HDD assumptions are too limiting.

I guess my hobby is weird, I bought that Jornada 820 in 1998 in order to listen to mp3s relocating from east coast to west coast and also use offline mapquest maps. Nostalgia! Also fired up the embedded visual tools in order to add a feature to my scrngrab tool. If I do a capture with the old way then the main app loses focus and the title bar becomes grey. Added an option for delayed capture to afford a bit of time to refocus the app. Took five lines of code to implement . #define NEWVAL 0x0102, case NEWVAL: Sleep(4000); DoCapture(); break;

Regretfully, I saw a message about failure to communicate with main battery last week and continuing into this week. Have seen threads here about re-celling batteries, but that's still scary to me.
 Top of the page
stingraze Page Icon Posted 2018-04-05 5:03 AM
#
Avatar image of stingraze
H/PC Vanguard

Posts:
3,656
Location:
Japan
Status:
Thank you for your detailed explanation.

The goal will be really awesome, good luck with it!

I think I first bought my Sigmarion (1) around 2002 or 3.
I bought it after I got into Windows CE by Cassiopeia E-700WE.
Since then, I've been on and off in this amazing form factor, but recently been interested in it a lot again.

Yes, recelling cells for me is scary too!
 Top of the page
smb_gaiden Page Icon Posted 2018-05-04 3:13 AM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Well, an update in case any are following along.

Added a linker option to tcc, so it will generate proper exe without the hex edit step. -Wl,subsysver=2.11 for example.

The bad news, when compiling tcc to be native to windows ce ran across a complication with ce loading the exe. Derived a smaller reproducible case where loader fails. Compiled with tcc executing off command line or explorer just hangs forever. Using remote process viewer shows that it never loaded.

Same code compiled with embedded visual tools runs well.

The two programs structure the PE files quite differently. And to double check, i disassembled the code sections from both exes. Both were valid. Next step was to hex edit the tcc generated exe and relayout the PE the way VS does. Still hangs by the loader. Since it hangs by the loader it cant get attached by the remote debugger.

Read some of the ce os source code for a later version and nothing showed a deadlock on a critical section or mutex based upon the loading of an exe with imports. In case the earlier version had such hangs on Waitfor*Object_ I tried the exe on my viewpad (CE3) and observed a hang. And by the way, what a difference in CPU speed between jornada820 and viewpad100!

Future items will be rechecking 4byte alignments, seeing if i can find loader code address on target and attach debugger, and finally deeper analysis of the ce os source code.

Anyone else have any paths to pursue for suggestions please?
 Top of the page
stingraze Page Icon Posted 2018-05-04 4:57 AM
#
Avatar image of stingraze
H/PC Vanguard

Posts:
3,656
Location:
Japan
Status:
Hm... too technical for me to give you advice over it, but I see lots of progress! :-)
 Top of the page
smb_gaiden Page Icon Posted 2018-05-05 4:08 AM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Quote
stingraze - 2018-05-03 8:57 PM

Hm... too technical for me to give you advice over it, but I see lots of progress! :-)


Thank you for the encouragement! Some progress, but lots of roadblocks. In order to feel more happenings in the progress category I got sidetracked today. In other threads have posted about receiving the Gemini PDA. Well, ported the TCC windows CE compiler to android (root required). Now I can make the same hit or miss Windows CE ARM executables from my Gemini PDA. Of course included the -Wl,-subsysver=2.11 or whichever CE version option in order to avoid hex editing the outputted binary.
 Top of the page
stingraze Page Icon Posted 2018-05-05 4:20 AM
#
Avatar image of stingraze
H/PC Vanguard

Posts:
3,656
Location:
Japan
Status:
Quote
smb_gaiden - 2018-05-05 1:08 PM

Quote
stingraze - 2018-05-03 8:57 PM

Hm... too technical for me to give you advice over it, but I see lots of progress! :-)


Thank you for the encouragement! Some progress, but lots of roadblocks. In order to feel more happenings in the progress category I got sidetracked today. In other threads have posted about receiving the Gemini PDA. Well, ported the TCC windows CE compiler to android (root required). Now I can make the same hit or miss Windows CE ARM executables from my Gemini PDA. Of course included the -Wl,-subsysver=2.11 or whichever CE version option in order to avoid hex editing the outputted binary.


Your sidetrack is great! Wow! TCC to Android w/ root! What's the minimum version of Android required for it to run?

Edited by stingraze 2018-05-05 4:20 AM
 Top of the page
smb_gaiden Page Icon Posted 2018-05-05 4:47 AM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Quote
stingraze - 2018-05-04 8:20 PM

Quote
smb_gaiden - 2018-05-05 1:08 PM

Quote
stingraze - 2018-05-03 8:57 PM

Hm... too technical for me to give you advice over it, but I see lots of progress! :-)


Thank you for the encouragement! Some progress, but lots of roadblocks. In order to feel more happenings in the progress category I got sidetracked today. In other threads have posted about receiving the Gemini PDA. Well, ported the TCC windows CE compiler to android (root required). Now I can make the same hit or miss Windows CE ARM executables from my Gemini PDA. Of course included the -Wl,-subsysver=2.11 or whichever CE version option in order to avoid hex editing the outputted binary.


Your sidetrack is great! Wow! TCC to Android w/ root! What's the minimum version of Android required for it to run?


Arbitrarily picked API level 18 which corresponds to the Android 4.3 APIs. The device I ran it on is well beyond that and sits at Android 7.1.1.

The workflow was:
1. Compile TCC
2. Compile TCC with cross-arm-wince
3. Create a coredll.def file for linking
4. Create a helloworld program
5. compile and link with -pie and -nostdlib and -lcoredll
6. Use a usb type C to USB adapter to host my CF card reader
7. copy the tcc generated exe to the CF card
8. safely eject card from android
9. put it in the HP jornada 820 and execute the app
10. success, have an IPA

More complex programs will likely meet the fate of the earlier posting. Still need to figure out what can hand the CE loader.

Edited by smb_gaiden 2018-05-05 4:49 AM
 Top of the page
stingraze Page Icon Posted 2018-05-05 7:53 AM
#
Avatar image of stingraze
H/PC Vanguard

Posts:
3,656
Location:
Japan
Status:
wow!

Excellent project progress!

I have never really thought of using modern Android as an compiler for Windows CE. Very interesting combination.
 Top of the page
C:Amie Page Icon Posted 2018-05-08 11:17 AM
#
Avatar image of C:Amie
Administrator
H/PC Oracle

Posts:
17,950
Location:
United Kingdom
Status:
Are you planning on supporting this and developing it through to being stable?
 Top of the page
smb_gaiden Page Icon Posted 2018-05-08 7:38 PM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Quote
C:Amie - 2018-05-08 3:17 AM

Are you planning on supporting this and developing it through to being stable?


If I can figure out what is hanging the ce loader then i will ask the tcc mailing list if i can mainline the changes. A couple dot versions back they claimed arm windows ce support, but looking att their PE structure for the binary they seem to have only dev-tested CE 4.0+. My only CE ARM devices are 2.11 and 3.0 and both of these hang with certain tcc generated binaries. Haven't found much discussion of using tcc to make ce exes online either.

Problem with CE architecture is that it isn't possible to do this:

pfn = (WndProc)GetProcAddress(hm, L"CreateProcess";

fwrite((void *)pfn, 1000, 1, fp);

objdump -b binary filecontent > file.s

Because CE doesn't actually have code at the address passed by GetProcAddress. They thunk into kernel functions via an exception table.

https://blogs.msdn.microsoft.com/ce_base/2006/02/02/inside-windows-c...

Pretty sure I have platform builder trial discs, but no device to target in order to try and see if expanded access that is afforded by kernel builder can help to solve.

So basically down to reading the CE shared source very carefully mixed with trial and error on laying out the PE format of the tcc generated exes.
 Top of the page
smb_gaiden Page Icon Posted 2018-05-09 5:59 AM
#
Avatar image of smb_gaiden
Factorite (Elite)

Posts:
212
Status:
Huge breakthrough happened just now. After 11th revision of hex editing binary generated by tcc; it works. Want to thank stingraze and CAmie for providing responses to this journey which spurred motivation to continue progress on this endeavor.

tldr; Hex edited tcc generated problem exe was made to be a non-problem and fully functional exe on "HP Jornada 820 Wince v2.11" (J8W211).

Today was a day of heavy experimentation:
1. Escalate ARM processor to 0x1F and try to breakpoint in kernel. Failed goal. Succeeded with the priv escalation and reading kernel memory and code sections. Failed to write to kernel code section to establish sw breakpoint.
2. Continue iterations of hex editing failed binary to keep getting closer to the eVC binary. Success, details below.

After 11 iterations of moving stuff around and resizing section data, the problem exe has finally been accepted, run, and not hung on J8W211. The alignment issue was likely culprit. Most of revisions 6 to 11 were about arbitrary 2, 4, and 8 byte alignments that are not documented anywhere. For revisions 1 - 5, there was a lot of re-ordering of data within sections and adding sections eVC had, but tcc did not.

The state we are in: Have a functional exe generated by tcc for wince 2.11 that requires very heavy exe editing (programatically edited and manually hex edited). Root cause of failure of tcc generated exe still unknown. Steps taken were to mimic the eVC generated binary as much as possible and incrementally did that until almost everything except the .text aka code section matched.

Future work:
1. Take a raw tcc generated exe and align stuff and hope it works. Then can just edit the src of tcc to align stuff as it spits out the exe.
2. if #1 fails, then edit tcc to account for missing sections, section ordering, and data within section and subsection alignments. Big work, but doable.
-. Learned a lot during this effort.
3. Between 1 & 2 there might be some experiments with hex editing various binaries from steps 1-11 of the experiment.


Don't get me wrong -- when I saw the last hex edited binary avoid hanging and properly MessageBoxW and printf out stuff .. I was simply amazed.
 Top of the page
C:Amie Page Icon Posted 2018-05-09 9:58 AM
#
Avatar image of C:Amie
Administrator
H/PC Oracle

Posts:
17,950
Location:
United Kingdom
Status:
Huge congratulations on persevering with this, you must have been sitting at your desk for days.

You're diving a lot lower than I usually care to go, so I take my hat off to you for it.

Will you document and release the PE structural information too as part of your efforts? Perhaps there are some people here with ARM/XScale 3.0 devices who could get some initial feedback there for you if you post some test binaries somewhere?
 Top of the page
1 2 3
Jump to forum:
Seconds to generate: 0.222 - Cached queries : 73 - Executed queries : 9