Vote Charlie!

Android rabbit hole: rsync and adb root

Posted at age 28.
Edited .

I have a Nexus 6P running Android 7.1.1, but I am a few updates behind. I haven’t gotten the process down for updating since my phone is rooted and I am wary of losing data, so I tend to delay. I almost documented what I did last time, but it turned out to be simpler than I expected and I neglected to save the instructions. Now I sat down to do it again, but figured I would first try to understand how to fully back up and restore the phone at more of a disk image level. It didn’t end as well as I hoped, and this entry serves more as a log of what I did than as a guide for anyone else.

Note: The process for installing fb-adb changed again as I explained in the second comment on Missing binaries for 10.12 on Brew #55:

@Manouchehri’s workaround no longer works due to https://github.com/Homebrew/homebrew-core/pull/11468:

➜  /tmp brew info android-ndk
Error: No available formula with the name "android-ndk"
It was migrated from homebrew/core to caskroom/cask/android-sdk.
You can access it again by running:
brew tap caskroom/cask
➜  /tmp brew cask info android-ndk
Error: No available Cask for android-ndk

New workaround

Install android-ndk after android-sdk using sdkmanager:

brew cask install android-sdk
sdkmanager android-ndk "platforms;android-19"

You also need these:

brew tap homebrew/dupes
brew install make python3

Then install fb-adb like this:

cd /tmp
git clone https://github.com/facebook/fb-adb.git
cd fb-adb
./autogen.sh
export ANDROID_SDK=/usr/local/share/android-sdk
export ANDROID_NDK=$ANDROID_SDK/ndk-bundle
mkdir build
cd build
../configure
/usr/local/bin/gmake
sudo cp -a fb-adb /usr/bin
rm -rf /tmp/fb-adb

To start, I wanted to switch from my clunky adb based backup procedure to an rsync method. I found a short blog post mentioning a single app you can install to set everything up. I installed SSHelper and was able to ssh to my phone and use rsync. Despite the SSH command using root@CHARLIE-PHONE, the logged in user was not root but u0_a58. I could su once logged in and then initiate rsync from the phone to my computer.

Logged into Android via SSH with SSHelper turned on, I could run the command:

rsync -avzH --delete --numeric-ids -e "ssh -i /data/data/com.arachnoid.sshelper/home/.ssh/id_rsa" --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/storage"} / cgorichanaz@CHARLIE-DESKTOP:/Volumes/Striped/Backups/Phones/2017-03-10_Nexus_6P

Note: The excludes are from Full system backup with rsync and perhaps aren’t quite right for Android. I did add the ,"/storage" because as far as I can tell, all user data is mounted to /storage/emulated/[usernumber] but actually resides in /data/media. I considered /config as well because I was getting errors like listed after this paragraph. CHARLIE-DESKTOPis mapped to my computer's IP address by my router, asCHARLIE-PHONE` is mapped to the phone’s IP address.

This works, but I would like to be able to initiate it from my computer. I also want to be able to rsync over USB instead of Wi-Fi if possible. I found the post How to use rsync over USB on Android with adb that explains how to install adb and copy a prebuilt rsync to Android, but it still won’t be able to run as root. A comment by Dwayne Litzenberger brought to my attention fb-adb, an improved adb interface, and suggested it could be used by rsync. I didn’t fully understand how this works, but I thought I’d try it before continuing to work on the “root” problem.

Installing fb-adb was not as simple as it claimed, probably due to Homebrew using an old version. When I got an error running brew install fb-adb, I tried to manually compile per the instructions. I ran configure, and later gmake, troubleshooted errors and repeated till it worked.

I ended up needing to install not just the Android command line tools but the entire Android Studio and Java SE Development Kit 8, as well as running the following commands:

brew tap homebrew/dupes
brew install homebrew/dupes/make
brew install android-ndk
brew install python3
android update sdk --no-ui --filter 'platform-tools'

Note: android-ndk was already installed by the attempted brew install fb-adb, but had I known that would fail ahead of time, I would need to install it per the preceding block of commands.

For reference (people Googling), here is some of the command line output:

Here is the procedure I would have followed to install fb-adb had I installed the prerequisites first:

cd /tmp
git clone https://github.com/facebook/fb-adb.git
cd fb-adb
./autogen.sh
export ANDROID_NDK=/usr/local/opt/android-ndk
export ANDROID_SDK=~/Library/Android/sdk
mkdir build
cd build
../configure
gmake
sudo cp -a fb-adb /usr/bin
rm -rf /tmp/fb-adb

I confirmed it was working using one of their sample commands, but I switched timestamp for date +%Y-%m-%d_%H-%M-%S:

fb-adb rcmd screencap -p > screenshot-$(date +%Y-%m-%d_%H-%M-%S).png

Now that I had fb-adb, I confirmed the fb-adb shell command was not magically giving me automatic root. I still tried the fancy syntax Litzenberger suggested, but it didn’t immediately work:

➜  Desktop rsync -avzH --delete --dry-run --numeric-ids -e "fb-adb shell" --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} :/storage/emulated/0/ /Volumes/Striped/Backups/Phones/2017-03-10_Nexus_6P/storage/emulated/0

/system/bin/sh: rsync: not found
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.2]

I think this makes sense since Android doesn’t have rsync, though it’s unclear to me if copying an rsync binary per the How to use rsync over USB on Android with adb article would yield anything different. From Android’s perspective, which rsync returns nothing unless rsync is in the path environment variable, I assume.

It seemed silly to install another rsync, but I didn’t want to investigate how the SSHelper instance is set up and whether I could link that to $PATH, especially if I might not need the rest of SSHelper. So I followed the brief tutorial in the article. Note I replaced /sdcard with /storage/emulated/0 per my phone’s filesystem.

brew install wget
wget -O rsync.bin http://github.com/pts/rsyncbin/raw/master/rsync.rsync4android
adb push rsync.bin /data/local/tmp/rsync
adb shell chmod 755 /data/local/tmp/rsync
adb shell 'exec >/storage/emulated/0/rsyncd.conf && echo address = 127.0.0.1 && echo port = 1873 && echo "[root]" && echo path = / && echo use chroot = false && echo read only = false'
# to run in foreground, requiring following commands in separate window:
adb shell /data/local/tmp/rsync --daemon --no-detach --config=/storage/emulated/0/rsyncd.conf --log-file=/proc/self/fd/2
# or to run in background:
# adb shell '/data/local/tmp/rsync --daemon --config=/storage/emulated/0/rsyncd.conf --log-file=/data/local/tmp/foo &'
adb forward tcp:6010 tcp:1873

Then I was able to use rsync like this:

rsync -avzH --numeric-ids --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} rsync://localhost:6010/root/ /Volumes/Striped/Backups/Phones/2017-03-10_Nexus_6P

Note: With this method, the previous root / is /root/ due to the module [root] specified in the config. Related information at Advanced applications of rsync.

It worked, but I did not have root and therefore got many permissions errors. On the StackExchange question rsync all files of remote machine over SSH without root user?, the answers include:

I found another StackExchange question, Permission denied using Rsync as root, that brought to my attention rsync’s uid and gid settings. I initially thought I could use them to get rsync to operate as root before realizing the rsync daemon would need to already be run as root before it could swith users, especially to switch to root! Just in case, I did a quick test by modifying the configuration file to include uid = root and gid = root:

adb shell 'exec >/storage/emulated/0/rsyncd.conf && echo address = 127.0.0.1 && echo port = 1873 && echo "[root]" && echo path = / && echo use chroot = false && echo read only = false && echo uid = root && echo gid = root'

Of course then rsync failed with the error @ERROR: setgid failed.

Perhaps it would be good enough to launch the rsync command from Android triggered by a call to adb shell or fb-adb shell, but as mentioned at the top, those do not allow directly logging in as root. That’s where I turned my attention next.

The StackExchange question Is there a way for me to run Adb shell as root without typing in ‘su’? has these answers:

  • Use Chainfire’s adbd insecure app

    I tried this, but it gave an error about failing to patch. At least the Play Store lets you instantly get a refund, whereas Apple lets developers rob you with no recourse even if the app description is a lie.

  • Use adb to execute script that elevates itself to root

    This is going to be my backup plan, as it is what I am already doing for part of my current backup scripts. (I don’t have my current ones on Github currently, but the concept is illustrated in an earlier draft of my Whatsapp backup script.) Matthew Read mentioned having a problem cleanly ending the adb shell session, which is perhaps why my backup scripts start with su root;exit. I’ll have to check if fb-adb fixes this.

  • Use su -c command

    I don’t remember if I tested this before, but it’s pretty similar to executing a script via adb.

  • ROM or boot.img modification required to launch adb as root or temporarily use an app

    Generally speaking, though, you need to pull your current boot.img from your phone, unpack it, extract the ramdisk, and find the default.prop file. This is a plaintext file, which you need to open in a text editor and then find the line that contains the value ro.secure. If the line says ro.secure=1 then you need to change it to ro.secure=0. After that you can re-pack the ramdisk and boot.img, then flash it to your phone. Once you reboot, you will be greeted with a # prompt whenever you perform adb shell without having to run su.

    Alternatively, if you are using a custom ROM but it doesn’t have this modification, you can just unzip the ROM and modify the boot.img that is included with it using the above steps. Then you can zip up the ROM with the newly modified boot.img and flash the zip file as you normally would.

    A comment on the answer suggested adb root, but that gave the error adbd cannot run as root in production builds. It seems the boot.img modification is my only path.

At this point it seems I should go the script route, but curiosity got the best of me, and I pursued the boot.img modification. I wanted to learn that better anyway. I’m not sure where the authoritative reference I should be using is, but I hoped Google would help me find what I needed quickly.

I found an article HOWTO: Unpack, Edit, and Re-Pack Boot Images, but it’s almost four years old and the cat /proc/mtd in the first step did not match my phone. The website also had some malwareesque popup ads. I therefore wasn’t sure I wanted to trust or look into the automated Perl scripts referenced therein.

A utility abootimg (or maybe this version?) can extract and pack Android boot images with a decent interface, so that seemed the way to go. I would first need to figure out how to get the image off my phone, or possibly just modify a new one I would install since I am trying to upgrade my Android anyway. But, I want to figure out my backup plan first, so I continued.

The StackExchange question How to pull boot.img and recovery.img from Sony Xperia E4 Dual? gives a partial procedure that looked promising, though again, the way the user got the partition information doesn’t match my phone. Apparently on Android 4.4.4 there is a /proc/dumchar_info and a bootimg partition mapped to /dev/block/mmcblk0. The user had a problem using dd to make a file, but as I commented, I think that was due to a math error. A comment by Matthew Read mentioned checking find /dev -name "by-name". That helped me determine my situation was boot maps to /dev/block/mmcblk0p34.

I looked for something listing the partition sizes and start sectors similar to what was in dumchar_info on Android 4.4.4 using grep:

angler:/ # grep -R mmcblk0p34 /proc
grep: /proc/sysrq-trigger: I/O error
/proc/partitions: 259        2      32768 mmcblk0p34
/proc/diskstats: 259       2 mmcblk0p34 1114 34790 287232 1400 0 0 0 0 0 1170 1400

Only the partitions file had a header:

angler:/ # head -1 /proc/partitions
major minor  #blocks  name
angler:/ # head -1 /proc/diskstats
   1       0 ram0 0 0 0 0 0 0 0 0 0 0 0

I am not familiar enough with these to be confident what all the numbers mean, and not wanting to get too far down the rabbit hole, I guessed I might not need to know the sizes at all, perhaps. I ran dd without:

angler:/ # dd if=/dev/block/mmcblk0p34 of=/storage/emulated/0/boot.img
65536+0 records in
65536+0 records out
33554432 bytes transferred in 0.543 secs (61794534 bytes/sec)

I transfered boot.img to my desktop and then installed abootimg:

adb pull /storage/emulated/0/boot.img ~/Desktop
cd /tmp
git clone https://github.com/ganadist/abootimg.git
cd abootimg
make -f Makefile.osx
sudo cp -a abootimg /bin

The info command seemed to work, so I think I successfully got the boot.img file.

➜  Desktop abootimg -i boot.img

Android Boot Image Info:

* file name = boot.img

* image size = 33554432 bytes (32.00 MB)
  page size  = 4096 bytes

* Boot Name = ""

* kernel size       = 11175493 bytes (10.66 MB)
  ramdisk size      = 1565964 bytes (1.49 MB)

* load addresses:
  kernel:       0x00008000
  ramdisk:      0x02000000
  tags:         0x01e00000

* cmdline = androidboot.hardware=angler androidboot.console=ttyHSL0 msm_rtb.filter=0x37 ehci-hcd.park=3 lpm_levels.sleep_disabled=1 boot_cpus=0-3 no_console_suspend buildvariant=user

* id = 0x2c728e9d 0x1d2a133b 0x3e7edc88 0xd4d73809 0xd628306a 0x00000000 0x00000000 0x00000000

I then extracted boot.img and then extracted d using a hint from another StackExchange question, but changing cpio -i to cpio -idmv to preserve timestamps, though the files ended up saying 1969 anyway.

➜ gunzip -c ../initrd.img | cpio -i
7608 blocks

My default.prop contained not only a ro.secure=1 line but also a ro.adb.secure=1 line. The StackExchange question How can I enable adbd during boot on Cyanogenmod? has an answer that mentions ro.adb.secure=0 disables “USB debugging requires authentication since Android 4.2.2”, while ro.secure=0 is for getting root. It is somewhat reassuring what I previously read was correct, though the way this user phrased it makes me wonder. He said “You won’t be able to get root (su)” without ro.secure=0, but hopefully he meant you won’t be able to launch adb as root, because I can already get root with su, I just can’t launch as root.

So I changed my ro.secure=1 to ro.secure=0, and looked for explicit instructions to repack the ramdisk file. The StackExchange question that showed me how to extract it doesn’t show the reverse. I am not horribly familiar with cpio, but I am guessing I can reverse a cpio -i with cpio -o. Certainty would be nice since this might brick my phone! Perhaps abootimg will throw an error if I mess up…

find . 2>/dev/null | cpio --quiet -c -o | gzip > ../initrd.img2

My new initrd.img ended up smaller than the original despite changing one character, but who knows what compression level was used before. Or maybe I did something wrong, ugh.

➜  tmp$ ls -al init*
-rw-r--r--  1 cgorichanaz  staff  1565964 Mar 12 18:25 initrd.img
-rw-r--r--  1 cgorichanaz  staff  1488431 Mar 12 18:53 initrd.img2

Ah, I found a source that seems better, Android: Boot image. I almost had it right, but they use the cpio flag -H to set a format of newc, which, according to the man page, is “The SVR4 portable cpio format”, whereas I had the flag -c that means a format of odc, which is “The old POSIX.1 portable octet-oriented cpio format.”:

➜  ramdisk$ find . | cpio -o -H newc | gzip > ../newramdisk.img
find: ./.subackup: Permission denied
6947 blocks

That “Permission denied” worries me, though, because it seems I might have piped that right into the archive? I ran again as root and also with the previous -c as a test:

sh-3.2# find . | cpio -o -H newc | gzip > ../newramdisk.img
7609 blocks

-rw-r--r--   1 cgorichanaz  staff   1565964 Mar 12 18:25 initrd.img
-rw-r--r--   1 cgorichanaz  staff   1488431 Mar 12 18:53 initrd.img2
-rw-r--r--   1 cgorichanaz  staff   1567545 Mar 12 19:03 newramdisk.img
-rw-r--r--   1 root         staff   1567303 Mar 12 19:04 newramdisk.img2

The correctly formatted file, newramdisk.img, is now much closer in size to the original initrd.img. I guess I’ll go ahead with packing it and flashing it.

➜  tmp$ abootimg -u boot.img -r newramdisk.img
reading ramdisk from newramdisk.img
Writing Boot Image boot.img

The resulting boot.img is exactly the same size as the original, 33554432 bytes. I pushed it to my device and overwrote the boot partition:

➜  tmp$ adb push boot.img /storage/emulated/0/
4577 KB/s (33554432 bytes in 7.158s)
➜  tmp$ adb shell
angler:/ $ su
angler:/ # dd if=/storage/emulated/0/boot.img of=/dev/block/mmcblk0p34
65536+0 records in
65536+0 records out
33554432 bytes transferred in 2.323 secs (14444439 bytes/sec)

Upon rebooting (eeks!), my device showed the Google logo for a few seconds and then loaded Team Win Recovery Project 3.0.2-2. Sometimes this loads when I restart, I think by some setting to allow me to get to it easily… but after selecting Reboot > System, I kept getting into the same place. I tried running adb shell and found it opened immediately to a root prompt. I think this means my modification worked for that aspect, though I cannot be sure since I never tried running adb shell with Team Win Recovery Project open instead of booted into Android.

I put the original boot.img back on the device and overwrote the partition again with dd, though this time /storage did not exist. I found /sdcard did exist, per the many examples I have seen. I had always used /storage/emulated/0 because my “File Manager HD” app shows that as the root path. Anyway, I was then able to boot back into Android, though without the ro.secure=0 modification I wanted.

After some more reading and experimenting, I discovered the Team Win Recovery Project boot software can backup the boot partition and restore from backup. I used this functionality to get the boot.img, though TWRP named it boot.emmc.win. I modified that using the above process with abootimg, and then put it back in place of the backup. I then restored from this modified backup, and was able to boot!

But, after loading, adb shell still loaded as a restricted user!

I found a forum post that explains my being about to su, but also seems to be saying I need a modified adb in addition or instead of modifying default.prop. Why is this so confusing!

“But,if it’s ro.secure=1, adb daemon is made to work in secure mode, and adb won’t change to root mode on issuing adb root command. However, if su binary is present in $PATH, u can still call su command from adb shell.”


6.Note that method 3,4 and 5 won’t work in Android 4.0 Jelly Bean onwards. According to Dan Rosenburg(drjbliss in XDA),the researcher who discovered adb root emulator exploit and many other exploits, Jelly Bean doesn’t parse any property files to set the ownership of adb daemon. The stock adbd will have to be replaced with an insecure one to gain adb root.

Following rather strange instructions from nijel8, I replaced adbd on my boot image and was able to boot back into Android. When I replaced sbin/adbd with adbd.17.png per the forum post, Android booted fine but the USB connection didn’t seem to register. I could not run adb commands, and the phone did not show it was connected to a computer. I then tried using the newer adbd.21.png from the APK, but phone still did not show it was connected by USB, but adb shell worked, and indeed went straight into root. But then after a while it didn’t even connect anymore:

$ adb shell
* daemon not running. starting it now on port 5037 *
cannot bind 'tcp:5037'
ADB server didn't ACK
* failed to start daemon *
error: cannot connect to daemon

At this point there are too many variables and old software or information to make this worth pursuing more. I haven’t found anyone with specific instructions with Nexus 6P or even Android 7.1 claiming to have gotten it working. I’ll just drop this for now and go to the script-on-device method!

Join the movement!

Show your support by signing up to be among the lucky few notified when I manage to blog. This could be as often as once a day or as infrequently as yearly!

blog comments powered by Disqus