Humpty Promotes
> Rfo-Basic
> Project Articles
>
Android NDK
binaries with rfo-Basic! (Sep
2015)
The 'hello world' moment
'hello world'
/həˈləʊ/
/wɜː(r)ld/ [noun]
[countable]
Desc: A wonderment that suspends you in time while future
possibilites fly around your head like flies.
Occurrence: rare.
The only other time I remember was the very first
output of my very first compiled language when a teenager.
Subsequent events fade into boredom and indifference. And that
wonderment gets forgotten, especially as languages and systems
become monotonously similar as the years drag on.
Surprisingly, the same feeling came back when executing a
native c-binary from a basic! app on my Android
phone.
'The
phone is running 'c' made code !' For a few brief
minutes, I was again a teenager.
I don't know wether it was because of my usual bias against
Java running android phones, or that I had forgotten that a
phone was really just a small sized PC, or that I had mentally
placed NDK as just another mass of geeky dependent-hell files
designed not to work for 'normal' people. But that the binary
actually ran, freaked me out and gave me quite a buzz for the
rest of the day.
This guide is simply about how to make a
native binary and run it from rfo-basic.
Preliminaries
It is assummed you are familiar with;
a) rfo-basic
b) the c language and compilers
c) OS terminals and scripts (windows or linux) and filenames.
In this guide, we will be using the c language and our host PC is
running linux. If you are using windows,
just convert the filepaths/names and script commands
accordingly.
What is
NDK?
The Android Native
Development Kit is primarily a system of building cross
compiled binaries of other languages to interface with Android's
OS (which runs on Java). It relies heavily on using JNI as an
interface to make library calls in-between the running binary and
Java. But as you might have noticed, this website (about
rfo-basic) avoids Java/XML due to it's complexities. So why use
NDK at all?
Why Use
It ?
It so happens that you can still compile NDK binaries and have
them execute
without JNI as long
as you don't call the Java libraries. Such needs are typically
tasks which are cpu intensive and don't need graphics or any other user
interaction. e.g Background or service utilties e.g
servers, batch file operations, or self contained utitlities
which have already been proven and written in other
languages.
Simple
Interface
All we need to do is to find a way to execute and perhaps devise a
simple way for the binary to talk to the basic!
program.
Launching is not a problem, Basic! already has a built-in shell
in the form of the system. command, of which we can
use to launch the
binary.
The next requirement is somewhere to put the binary,
where we can give it permissions to execute.
All android apps are entitled a private area of storage called
Primary Internal Private
Storage. This is usually at location /data/data/<package_name>.
It is created when the app is installed. The app has full rights
here, so putting the binary here means we can give it the
right to
execute. Another bonus, is that the directory is wiped
when the app is uninstalled.
For communication between the binary and basic!, you can either
use a protocol of read/write to temporary files, or alternatively
the stdin/stdout of the shell/binary.
Make a
Binary
But first things first, let's see how we can build an actual
binary with NDK.
The development environment is more simple than you think. There
is no special IDE, no drivers to install and you can install to
anywhere you want. You don't even have to set any environment
variables if you don't want. Configuration It is mostly setting
values inside provided scripts.
1. Download the latest NDK
distribution.
The NDK distro is a gigantic collection of compilers and
toolchains (utilities for compilation) and examples that cater
for windows, linux and mac.
Pick the distro that matches your development PC.
Download and run the self-extracter which will extract a tree
of files to a directory of your choice.
2. Check for build script ndk-build.
In the main directory there is script file called 'ndk-build'
This is the main script to call for compilation. Note it's
location, you will need it for later.
(For windows, you can use 'ndk-build.cmd' otherwise you
need to install cygwin to be able to use the linux script)
3. Navigate to the standalone example file.
Open a terminal and cd into tests/standalone/basic-c-compile/jni/
(although we won't use jni, this is the standard
sub-directory structure.)
This is where the source files and makefile go, and where we
will be operating from.
You will find a file called
main.c which is the 'hello
world' c example.
#include
<stdio.h>
int
main()
{
printf
("hello world\n");
return
0;
}
Now we check if a makefile is here..
4. Create Android.mk
Below is the makefile. If one is not provided, create this one
(infact, you should probably use this one);
Android.mk
LOCAL_PATH:=
$(call my-dir)
include
$(CLEAR_VARS)
LOCAL_MODULE :=
main
#APP_PLATFORM
:= android-16
#LOCAL_CFLAGS
+= -fPIE
#LOCAL_LDFLAGS
+= -fPIE -pie
LOCAL_SRC_FILES:=
main.c
include
$(BUILD_EXECUTABLE)
This makefile takes one source file main.c, it will compile and
link it with android's bionic library (a cut
down version of the standard c library for android),
and generate a binary called main.
In the middle of the makefile are 3 commented lines. The 3 lines
determine wether PIE is turned on or off.
#APP_PLATFORM :=
android-16
#LOCAL_CFLAGS +=
-fPIE
#LOCAL_LDFLAGS += -fPIE
-pie
#commented =
PIE disabled ( off
)
un-commented = PIE enabled ( on )
If your Android phone version is JellyBean or later (4.1+) i.e api16+,
then un-comment the
lines (PIE on).
If your Android phone version is older, i.e is less than
4.1, then #comment
the lines out (PIE Off)
(see later for explanation of PIE)
5. Create a compile script.
In this directory, create a script called
go.sh (or go.cmd) and type in
the path to the main build script ndk-build. e.g
go.sh
/mnt/sda4/dev/android-ndk-r10d/ndk-build
This
means you can run this script instead of setting up
environment variables with a path.
You will run '
go.sh' whenever you need to
compile in this directory.
6. Compile
Run the compile script ./go.sh
This should generate any executables inside ../libs/armeabi/
i.e ../libs/armeabi/main
(It will also generate some files in
../obj/local/armeabi/ but we aren't interested in
those.)
main is the binary
executable which you will copy to your phone (see later).
Your Basic!
project
For editor mode, the package_name is "com.rfo.basic".
For Apk, the package name is whatever you named your
package(with your apk builder),
For both cases, the project directory will be /mnt/sdcard/<package_name>/
1. Copy the binary
you compiled to your <>/data directory. e.g
/mnt/sdcard/rfo-basic/data/main
2. In your program, initialise the basic shell and go to private
storage /data/data/<package_name>
p$="com.rfo.basic"
% package
name
path$ = "/data/data/"+p$+"/"
system.open
system.write "cd "+ path$+"
2>&1"
pause 50
3. Copy over the binary to private storage
b$="main"
% binary
name
file.root datapath$
%
<base>/data
srce$ =
datapath$+"/"+b$
system.write
("cat "+srce$+" >./"+b$+" 2>&1")
note: the binary should first be in <base>/data.
If making a package either the apk-builder should ensure this
or your basic program should pre-copy this from assets to
<base> /data.
4. Make the binary executable
system.write "chmod 777
"+b$+"
2>&1" %
make executable
5. Run It.
system.write
"./"+b$+"
2>&1" % execute
binary
6. Get the Output
r$=""
%
assume no reponse
pause
1000
system.READ.READY
ready % if response
while
ready
system.READ.LINE l$
r$ += l$+"\n"
system.READ.READY ready
repeat
print r$
% output
7. Cleanup and close the shell
onBackKey:
system.close
system.open
system.write "cd "+ path$ + ";rm "+b$
% delete the
binary
pause
100
system.close
end
When you run this basic program, it should copy over and execute
the binary in a shell, then spool any output to the screen. Any
shell errors will normally show by the spooling. If all goes
well, you will see "hello world" .
That's It !
About the
Shell
Basic! can execute binaries because of it's system command. This is a shell
by which you can pass commands to. You can only open one shell at
a time but even so, it is a powerful tool.
Process
Properties
The shell is a truly spawned parallel process.
The executed binary is also a truly spawned parallel
process.
Basic, the shell and the binary will all have differing
PIDs.
Both the shell and the binary might be seen under 'Basic' in a
task manager but the shell process is named "sh" and the binary process is
named with it's called command e.g "./main"
Conditions for exit
If Basic exits or ends, it will also close the shell (if still
open).
If the shell is closed, any spawned binaries will continue to
run.
If Basic crashes, then both the shell and the binaries may
continue executing.
Communication
Because the binary is forked from the shell, they will share
the same environment. This means stdin/stdout will be
shared.
Data from
binary/shell to Basic.
Any data sent to stdout will set system.read.ready
true after a
linefeed (line based).
If Basic does not read the data, any furthur data will be
buffered.
The binary may need to flush it's output stream to ensure
this.
If the shell or binary has nothing to send, (or any sent data
is not yet flushed),
and any previous data has already been read then system.read.ready will be
false,
Regardless of any condition of system.read.ready, Basic will
always continue to the next instruction (will not be
blocked).
Data from Basic
to binary/shell.
Any system.write
will be sent to stdin.
If the shell or binary is not reading stdin, the data will be
buffered and Basic will continue to the next instruction
(will not be blocked).
If the binary is reading from stdin and there is no data sent
(from Basic), then the binary will be blocked.
About PIE
In Jun 2012, Google began to enable PIE (Position
Independent Executables) in Android as a security measure.
They sneaked this into the OS starting with JellyBean (4.1) and
ending with KitKat (4.4). This was the 'grace period' whereby any
phone within this range is able to execute both PIE and non-PIE
compiled code. Any phone after this range (starting with Lollipop
(5.0)) will only execute PIE binaries.
This means older phones (< v4.1) don't work with PIE while
newer phones (>4.4) don't work without it.
The only choice left for binary makers then, was either to force
the customer to upgrade their phones or make sure the package
ships with both types of binaries. For the latter case, the app
will need to choose which binary to execute at runtime.
Fortunately in Basic, you can get the OS version like this
a=0
device a
bundle.get a, "OS",
s$
With a bit of manipulation, you can convert this to an
integer value to decide which binary to use.
(For apk making, you will need permission READ_PHONE_STATE for the 'device'
command)
Copy over the binary to private storage matching the OS
pie or no-pie version. (modify the program above like this;)
b$="main"
%
binary name
file.root datapath$
%
<base>/data
a=0
device
a
bundle.get a, "OS",
s$
v=
val(word$(s$,1,"\\."))*10+val(word$(s$,2,"\\."))
if v<41 then
suffix$="-nopie" else suffix$="-pie"
srce$ =
datapath$+"/"+b$+suffix$
system.write
("cat "+srce$+" >./"+b$+"
2>&1")
|
Note that the final binary copied to
private storage need not have the pie/nopie suffix.
Ofcourse this means you have to compile two versions of your
binary, main-pie and
main-nopie.
Android.mk can build both versions at the same time.
Android.mk
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := main-nopie
LOCAL_SRC_FILES:= main.c
include $(BUILD_EXECUTABLE)
#------------------------------
include $(CLEAR_VARS)
LOCAL_MODULE := main-pie
LOCAL_SRC_FILES:= main.c
APP_PLATFORM := android-16
LOCAL_CFLAGS += -fPIE
LOCAL_LDFLAGS += -fPIE -pie
include $(BUILD_EXECUTABLE) |
Output
$ go.sh
[armeabi] Compile thumb :
main-nopie <= main.c
[armeabi]
Executable :
main-nopie
[armeabi]
Install :
main-nopie => libs/armeabi/main-nopie
[armeabi] Compile
thumb : main-pie <= main.c
[armeabi]
Executable :
main-pie
[armeabi]
Install :
main-pie => libs/armeabi/main-pie
|
note: since NDK 10d:
PIE is supposed to be anabled by default if APP_PLATFORM :=
android-16
but experience suggests this is not enough. To be sure that PIE
is enabled, also set :
LOCAL_CFLAGS += -fPIE
LOCAL_LDFLAGS += -fPIE -pie
-End.
Sources and Acknowledgements:
https://en.wikipedia.org/wiki/Android_version_history
https://www.duosecurity.com/blog/exploit-mitigations-in-android-jelly-bean-4-1
http://stackoverflow.com/questions/24818902/running-a-native-library-on-android-l-error-only-position-independent-executab
Support my projects!
Donate
