Difference between revisions of "Updating extensions for iOS 9"

From iPhone Development Wiki
Jump to: navigation, search
(canOpenURL restrictions)
m (Formatting.)
Line 7: Line 7:
 
If you want to see what's been recently updated on this page, you can use the wiki's [http://iphonedevwiki.net/index.php?title=Updating_extensions_for_iOS_9&action=history history feature] to compare the revisions (to look at the diff) since the last time you visited this page.
 
If you want to see what's been recently updated on this page, you can use the wiki's [http://iphonedevwiki.net/index.php?title=Updating_extensions_for_iOS_9&action=history history feature] to compare the revisions (to look at the diff) since the last time you visited this page.
  
== Compiling ldid on El Capitan ==
+
= Compiling ldid on El Capitan =
  
 
ldid [https://github.com/Homebrew/homebrew/commit/ac8d9201b8ae5ffe2d6aef6acf08cf1f654caccc was recently added to] the main Homebrew repo.
 
ldid [https://github.com/Homebrew/homebrew/commit/ac8d9201b8ae5ffe2d6aef6acf08cf1f654caccc was recently added to] the main Homebrew repo.
Line 16: Line 16:
 
This is more convenient for ensuring ldid is kept up to date in future.
 
This is more convenient for ensuring ldid is kept up to date in future.
  
== What has changed in iOS 9? (Classes, frameworks, etc.) ==
+
= What has changed in iOS 9? (Classes, frameworks, etc.) =
  
=== Compilation changes ===
+
== Compilation changes ==
  
 
32 bit binaries loaded on 64 bit devices fail to do so since the 32 bit pagesize has been changed from 4096 bytes to 16384 bytes.
 
32 bit binaries loaded on 64 bit devices fail to do so since the 32 bit pagesize has been changed from 4096 bytes to 16384 bytes.
Line 40: Line 40:
 
One example of this are tweaks that modify Cydia, which is a 32 bit app.
 
One example of this are tweaks that modify Cydia, which is a 32 bit app.
  
=== Entitlements ===
+
== Entitlements ==
  
''Every'' dylib meant for injection has to be signed to work on iOS, even if no entitlements are required. Please make sure that your toolchain of choice is producing signed dylibs, if it is a fat binary, make sure that ''all'' slices are signed.
+
'''''Every''''' dylib meant for injection has to be signed to work on iOS, even if no entitlements are required. Please make sure that your toolchain of choice is producing signed dylibs, if it is a fat binary, make sure that ''all'' slices are signed.
  
 
Use ldid to sign:
 
Use ldid to sign:
Line 55: Line 55:
 
</source>
 
</source>
  
==== Granting them at runtime ====
+
=== Granting them at runtime ===
  
 
To grant entitlements to a specific process in iOS 9, it seems that iOS 8's _BSAuditTokenTaskHasEntitlement function in assertiond no longer does the trick, the new _BSXPCConnectionHasEntitlement needs to be hooked instead.
 
To grant entitlements to a specific process in iOS 9, it seems that iOS 8's _BSAuditTokenTaskHasEntitlement function in assertiond no longer does the trick, the new _BSXPCConnectionHasEntitlement needs to be hooked instead.
  
=== Sandbox Restrictions ===
+
== Sandbox Restrictions ==
 +
 
 
Tweaks that create or edit files from a sandbox application outside the app's container is no longer allowed
 
Tweaks that create or edit files from a sandbox application outside the app's container is no longer allowed
 
*  Use an XPC method to communicate with SpringBoard from the sandbox application
 
*  Use an XPC method to communicate with SpringBoard from the sandbox application
Line 71: Line 72:
 
* After the v1.1 Pangu Untether update it is no longer possible to save/create/modify the preferences from Sandboxed applications in "atomically" mode.
 
* After the v1.1 Pangu Untether update it is no longer possible to save/create/modify the preferences from Sandboxed applications in "atomically" mode.
 
You will get something like this:
 
You will get something like this:
 +
 
<source lang="text">
 
<source lang="text">
 
Sandbox: processname(PID) deny(1) file-write-unlink/file-write-create  /private/var/mobile/Library/Preferences/prefs_file_name.plist
 
Sandbox: processname(PID) deny(1) file-write-unlink/file-write-create  /private/var/mobile/Library/Preferences/prefs_file_name.plist
 
</source>
 
</source>
 +
 
As a workaround you can just replace "atomically:YES" with "atomically:NO":
 
As a workaround you can just replace "atomically:YES" with "atomically:NO":
 +
 
<source lang="objc">
 
<source lang="objc">
 
[prefsDict writeToFile:settingsFilePath atomically:NO];
 
[prefsDict writeToFile:settingsFilePath atomically:NO];
Line 81: Line 85:
 
* some sysctl calls and proc_* functions cannot be used in a sandbox now
 
* some sysctl calls and proc_* functions cannot be used in a sandbox now
 
trying to use these functions in a sandboxed app will throw an error like:
 
trying to use these functions in a sandboxed app will throw an error like:
 +
 
<source lang="text">
 
<source lang="text">
 
Sandbox: [PROCCESSNAME] deny(1) process-info-listpids
 
Sandbox: [PROCCESSNAME] deny(1) process-info-listpids
 
</source>
 
</source>
  
=== UIScreen changes (unfinished section) ===
+
== UIScreen changes (unfinished section) ==
  
 
Specifically iPads now seems to have been broken when using UIScreen bounds and does not seem to take orientation into account. Please add solution to here.
 
Specifically iPads now seems to have been broken when using UIScreen bounds and does not seem to take orientation into account. Please add solution to here.
  
== Which tools and other preexisting things are still working on iOS 9? Which ones don't work? ==
+
= Which tools and other preexisting things are still working on iOS 9? Which ones don't work? =
  
 
No fixes for the following at the time of this writing. Note that these work on 32-bit devices, such as an iPhone 5.
 
No fixes for the following at the time of this writing. Note that these work on 32-bit devices, such as an iPhone 5.
Line 119: Line 124:
 
This occurs due to the change in the 32-bit pagesize on 64-bit CPUs in iOS 9. The libraries noted above need to be rebuilt with "-Wl,-segalign,4000".
 
This occurs due to the change in the 32-bit pagesize on 64-bit CPUs in iOS 9. The libraries noted above need to be rebuilt with "-Wl,-segalign,4000".
  
=== Killed: 9 ===
+
== Killed: 9 ==
  
 
Pangu9 causes many command-line tools to not work, with the error "Killed: 9"
 
Pangu9 causes many command-line tools to not work, with the error "Killed: 9"
Line 125: Line 130:
 
This can be solved by running "ldid -S `which <command>`"
 
This can be solved by running "ldid -S `which <command>`"
  
=== Daemons ===
+
== Daemons ==
  
 
In iOS 9 the way daemons are loaded appears to have changed. Daemons prefixed with "com.apple" are loaded first with other daemons being loaded by launchd significantly later. This creates a bug for daemons that use XPC to communicate with SpringBoard. SpringBoard will be loaded before the daemon meaning a connection can never be established. Changing the daemon prefix to "com.apple" appears to make it load at the same time as SpringBoard allowing for the connection to succeed. More research is required into why other daemons are being loaded much later than in iOS 8.
 
In iOS 9 the way daemons are loaded appears to have changed. Daemons prefixed with "com.apple" are loaded first with other daemons being loaded by launchd significantly later. This creates a bug for daemons that use XPC to communicate with SpringBoard. SpringBoard will be loaded before the daemon meaning a connection can never be established. Changing the daemon prefix to "com.apple" appears to make it load at the same time as SpringBoard allowing for the connection to succeed. More research is required into why other daemons are being loaded much later than in iOS 8.
  
 
Additionally, daemons are now outputting the error:
 
Additionally, daemons are now outputting the error:
 +
 
<source lang="text">
 
<source lang="text">
 
This daemon is not allowed to execute. Running anyway.
 
This daemon is not allowed to execute. Running anyway.
Line 137: Line 143:
  
 
Daemons that use more than 5MB are killed via Jetsam:
 
Daemons that use more than 5MB are killed via Jetsam:
 +
 
<source lang="text">
 
<source lang="text">
 
Oct 28 12:24:29 My-iPhone-6s ReportCrash[13169] <Notice>: MS:Notice: Injecting: (null) [ReportCrash] (1240.10)
 
Oct 28 12:24:29 My-iPhone-6s ReportCrash[13169] <Notice>: MS:Notice: Injecting: (null) [ReportCrash] (1240.10)
Line 149: Line 156:
 
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13172] <Warning>: report not saved because the limit of 25 for 298 logs has been reached.
 
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13172] <Warning>: report not saved because the limit of 25 for 298 logs has been reached.
 
</source>
 
</source>
 +
 
The Jetsam log is written to ~mobile/Library/Logs/CrashReporter however it doesnt contain anything useful. It's possible Jetsam properties are required to be added to the plist to raise the daemon's memory limit. Apples launch daemon plists use properties in a device specific globals file at /System/Library/LaunchDaemons/com.apple.jetsamproperties.N71.plist
 
The Jetsam log is written to ~mobile/Library/Logs/CrashReporter however it doesnt contain anything useful. It's possible Jetsam properties are required to be added to the plist to raise the daemon's memory limit. Apples launch daemon plists use properties in a device specific globals file at /System/Library/LaunchDaemons/com.apple.jetsamproperties.N71.plist
 
It's not clear if you can still put the JetsamProperties keys in your daemon plist or if you need to add to the globals list, some Apple ones still have them e.g. com.apple.ac.plist.
 
It's not clear if you can still put the JetsamProperties keys in your daemon plist or if you need to add to the globals list, some Apple ones still have them e.g. com.apple.ac.plist.
 +
 
I've tried adding overrides for custom daemons but it has no effect, still are killed when 5MB is reached.
 
I've tried adding overrides for custom daemons but it has no effect, still are killed when 5MB is reached.
 
Tweaks that erroneously add UIKit to daemons will cause them to be killed since they unexpectedly will go over 5MB.
 
Tweaks that erroneously add UIKit to daemons will cause them to be killed since they unexpectedly will go over 5MB.
  
=== Connecting to UNIX sockets ===
+
== Connecting to UNIX sockets ==
 +
 
 
Tweaks built with a library injected into an app, communicating to a daemon using a UNIX a socket, might fail to connect to the UNIX socket, with error code EPERM, and the following syslog message:
 
Tweaks built with a library injected into an app, communicating to a daemon using a UNIX a socket, might fail to connect to the UNIX socket, with error code EPERM, and the following syslog message:
kernel[0] <Notice>: Sandbox: app-name(<pid>) deny(1) network-outbound /private/var/tmp/mysocket
+
 
 +
    kernel[0] <Notice>: Sandbox: app-name(<pid>) deny(1) network-outbound /private/var/tmp/mysocket
 +
 
 
Note that this worked with the original Pangu untether and has been failing to work (as described) with the latest (1.1.0) Pangu Fuxi Qin.
 
Note that this worked with the original Pangu untether and has been failing to work (as described) with the latest (1.1.0) Pangu Fuxi Qin.
  
Line 163: Line 175:
 
A current workaround is to place the UNIX socket inside the app's sandbox.
 
A current workaround is to place the UNIX socket inside the app's sandbox.
  
=== Extension does not have filter ===
+
== Extension does not have filter ==
On iOS 9, tweaks '''must''' specify a [[Cydia_Substrate#MobileLoader|MobileLoader filter]] or Substrate will prevent the tweak from getting injected, with the following error:
+
 
 +
Starting on 0.9.6100 version of CydiaSubstrate, tweaks '''''must''''' specify a [[Cydia_Substrate#MobileLoader|MobileLoader filter]] or Substrate will prevent the tweak from getting injected at all, with the following error:
 +
 
 
<source lang="text">
 
<source lang="text">
 
MS:Error: extension does not have filter
 
MS:Error: extension does not have filter
 
</source>
 
</source>
  
=== canOpenURL restrictions ===
+
== canOpenURL restrictions ==
 +
 
 
If you have a tweak that relies on canOpenURL it might not work because now the URLs that are allowed to be checked are required to be specified in the host app's Info.plist under LSApplicationQueriesSchemes. Unfortunately editing this list does not work because it appears to be checked at installation time, and also it can only be called with 50 URLs, once that limit is reached it fails regardless of any edits to the list. It's currently unknown where the database is stored on the device.
 
If you have a tweak that relies on canOpenURL it might not work because now the URLs that are allowed to be checked are required to be specified in the host app's Info.plist under LSApplicationQueriesSchemes. Unfortunately editing this list does not work because it appears to be checked at installation time, and also it can only be called with 50 URLs, once that limit is reached it fails regardless of any edits to the list. It's currently unknown where the database is stored on the device.
  

Revision as of 18:01, 21 December 2015

Let's collect knowledge like we did with Updating extensions for iOS 8 and Updating extensions for iOS 7 - paste in your notes and share what you've learned, and somebody else will organize it later. :) If you want to ask questions and share tips over chat with other developers, see How to use IRC for how to connect to #theos and #iphonedev.

Hey developer, you can add your knowledge here! Yes, you! Make an account and edit this page!

It's also helpful to double-check the statements here and add more info! These are notes and drafts from early research - feel free to update them.

If you want to see what's been recently updated on this page, you can use the wiki's history feature to compare the revisions (to look at the diff) since the last time you visited this page.

Compiling ldid on El Capitan

ldid was recently added to the main Homebrew repo.

brew update
brew install ldid

This is more convenient for ensuring ldid is kept up to date in future.

What has changed in iOS 9? (Classes, frameworks, etc.)

Compilation changes

32 bit binaries loaded on 64 bit devices fail to do so since the 32 bit pagesize has been changed from 4096 bytes to 16384 bytes.

Tweaks targeted at 32 bit binaries on iOS 9 must now be compiled with

   -Wl,-segalign,4000

This LDFLAG can be used to compile for older iOS versions as it had to be a multiple of 1000 and this new alignment is compatible.

If using Theos, add it like so to your makefile:

   XXX_LDFLAGS += -Wl,-segalign,4000

This fix is integrated with kirb/theos. (Be sure to make update-theos regularly.)

If using Xcode, add a new entry to Other linker flags containing "-Wl,-segalign,4000" to the build settings of your project or target and make sure that the build option "Enable Bitcode" is disabled.

Source: saurik's tweet

One example of this are tweaks that modify Cydia, which is a 32 bit app.

Entitlements

Every dylib meant for injection has to be signed to work on iOS, even if no entitlements are required. Please make sure that your toolchain of choice is producing signed dylibs, if it is a fat binary, make sure that all slices are signed.

Use ldid to sign:

   ldid -S Tweak.dylib

Failure to do this will invalidate the process and make it lose all entitlements. The standard symptom is the following, but frankly, it is confusing why any binaries are in the wild that haven't at least been passed through ldid, so please don't rely on this symptom and just fix your build environment.

xbs/Sources/BackBoardServices/SpringBoard-3296.10.2/megatrond/SystemAppService/BKSSystemApplicationClient.m:32
Oct 14 21:29:57 iPhone SpringBoard[1860] <Error>: *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Client lacks entitlement com.apple.backboard.client’

Granting them at runtime

To grant entitlements to a specific process in iOS 9, it seems that iOS 8's _BSAuditTokenTaskHasEntitlement function in assertiond no longer does the trick, the new _BSXPCConnectionHasEntitlement needs to be hooked instead.

Sandbox Restrictions

Tweaks that create or edit files from a sandbox application outside the app's container is no longer allowed

  • Use an XPC method to communicate with SpringBoard from the sandbox application

See CPDistributedMessagingCenter for some example code.

This way you could communicate with a SpringBoard class to get it to save or create your files

You would need to add AppSupport framework in your makefile

   XXX_FRAMEWORKS = AppSupport
  • After the v1.1 Pangu Untether update it is no longer possible to save/create/modify the preferences from Sandboxed applications in "atomically" mode.

You will get something like this:

Sandbox: processname(PID) deny(1) file-write-unlink/file-write-create  /private/var/mobile/Library/Preferences/prefs_file_name.plist

As a workaround you can just replace "atomically:YES" with "atomically:NO":

[prefsDict writeToFile:settingsFilePath atomically:NO];
  • some sysctl calls and proc_* functions cannot be used in a sandbox now

trying to use these functions in a sandboxed app will throw an error like:

Sandbox: [PROCCESSNAME] deny(1) process-info-listpids

UIScreen changes (unfinished section)

Specifically iPads now seems to have been broken when using UIScreen bounds and does not seem to take orientation into account. Please add solution to here.

Which tools and other preexisting things are still working on iOS 9? Which ones don't work?

No fixes for the following at the time of this writing. Note that these work on 32-bit devices, such as an iPhone 5.

  • Cycript works as of 28/10/2015 version 0.9.503
  • the official python from cydia may fail however due to the work of Linus Yang and reddit user ryley_angus we have a working binary that works on iOS 9 !

install all packages from [1], most things I tested worked.

  • python fails with the following error (the one on cydia)
dyld: Library not loaded: /usr/lib/libpython2.7.dylib
  Referenced from: /usr/bin/python
  Reason: no suitable image found.  Did find:
	/usr/lib/libpython2.7.dylib: mmap() error 22 at address=0x00242000, size=0x0002A000 segment=__DATA in Segment::map() mapping /usr/lib/libpython2.7.dylib
	/usr/lib/libpython2.7.dylib: mmap() error 22 at address=0x003D6000, size=0x0002A000 segment=__DATA in Segment::map() mapping /usr/lib/libpython2.7.dylib
Trace/BPT trap: 5
  • As of (unknown date) in version 1.4.18-7, lighttpd works on iOS 9.
  • ruby fails with the following error
dyld: lazy symbol binding failed: re-exported symbol 'unknown' not found for image libgcc_s.1.dylib expected re-exported in libSystem.B.dylib, node=0x3980b148

This occurs due to the change in the 32-bit pagesize on 64-bit CPUs in iOS 9. The libraries noted above need to be rebuilt with "-Wl,-segalign,4000".

Killed: 9

Pangu9 causes many command-line tools to not work, with the error "Killed: 9"

This can be solved by running "ldid -S `which <command>`"

Daemons

In iOS 9 the way daemons are loaded appears to have changed. Daemons prefixed with "com.apple" are loaded first with other daemons being loaded by launchd significantly later. This creates a bug for daemons that use XPC to communicate with SpringBoard. SpringBoard will be loaded before the daemon meaning a connection can never be established. Changing the daemon prefix to "com.apple" appears to make it load at the same time as SpringBoard allowing for the connection to succeed. More research is required into why other daemons are being loaded much later than in iOS 8.

Additionally, daemons are now outputting the error:

This daemon is not allowed to execute. Running anyway.

This can be fixed by adding the plist entry ExecuteAllowed with a boolean YES.

Daemons that use more than 5MB are killed via Jetsam:

Oct 28 12:24:29 My-iPhone-6s ReportCrash[13169] <Notice>: MS:Notice: Injecting: (null) [ReportCrash] (1240.10)
Oct 28 12:24:29 My-iPhone-6s ReportCrash[13169] <Notice>: MS:Notice: Loading: /Library/MobileSubstrate/DynamicLibraries/RocketBootstrap.dylib
Oct 28 12:24:30 My-iPhone-6s diagnosticd[209] <Error>: error evaluating process info - pid: 13166, punique: 13166
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13169] <Notice>: Formulating report for process[13166] pmpd
Oct 28 12:24:30 My-iPhone-6s com.apple.xpc.launchd[1] (org.protectmyprivacy.pmpd) <Notice>: Service only ran for 0 seconds. Pushing respawn out by 10 seconds.
Oct 28 12:24:30 My-iPhone-6s UserEventAgent[128] <Notice>: jetsam: kernel termination snapshot being created
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13169] <Warning>: report not saved because it is non-actionable
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13172] <Notice>: MS:Notice: Injecting: (null) [ReportCrash] (1240.10)
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13172] <Notice>: MS:Notice: Loading: /Library/MobileSubstrate/DynamicLibraries/RocketBootstrap.dylib
Oct 28 12:24:30 My-iPhone-6s ReportCrash[13172] <Warning>: report not saved because the limit of 25 for 298 logs has been reached.

The Jetsam log is written to ~mobile/Library/Logs/CrashReporter however it doesnt contain anything useful. It's possible Jetsam properties are required to be added to the plist to raise the daemon's memory limit. Apples launch daemon plists use properties in a device specific globals file at /System/Library/LaunchDaemons/com.apple.jetsamproperties.N71.plist It's not clear if you can still put the JetsamProperties keys in your daemon plist or if you need to add to the globals list, some Apple ones still have them e.g. com.apple.ac.plist.

I've tried adding overrides for custom daemons but it has no effect, still are killed when 5MB is reached. Tweaks that erroneously add UIKit to daemons will cause them to be killed since they unexpectedly will go over 5MB.

Connecting to UNIX sockets

Tweaks built with a library injected into an app, communicating to a daemon using a UNIX a socket, might fail to connect to the UNIX socket, with error code EPERM, and the following syslog message:

   kernel[0] <Notice>: Sandbox: app-name(<pid>) deny(1) network-outbound /private/var/tmp/mysocket

Note that this worked with the original Pangu untether and has been failing to work (as described) with the latest (1.1.0) Pangu Fuxi Qin.

Update: This seems to be untrue. This doesn't work with the original Pangu untether as well. Maybe the Cydia update process has to do something with it? Maybe stashing?

A current workaround is to place the UNIX socket inside the app's sandbox.

Extension does not have filter

Starting on 0.9.6100 version of CydiaSubstrate, tweaks must specify a MobileLoader filter or Substrate will prevent the tweak from getting injected at all, with the following error:

MS:Error: extension does not have filter

canOpenURL restrictions

If you have a tweak that relies on canOpenURL it might not work because now the URLs that are allowed to be checked are required to be specified in the host app's Info.plist under LSApplicationQueriesSchemes. Unfortunately editing this list does not work because it appears to be checked at installation time, and also it can only be called with 50 URLs, once that limit is reached it fails regardless of any edits to the list. It's currently unknown where the database is stored on the device.

iOS 9 Launch scheme approval where it asks for permission to open an app: /private/var/preferences/com.apple.launchservices.schemeapproval.plist

For error-free access to this data query the canOpenURL: status in a daemon with the appropriate entitlements or inside SpringBoard using its entitlements and make the specific query accessible over IPC using RocketBootstrap, darwin_set_state or similar if necessary.