Difference between revisions of "Dyld shared cache"

From iPhone Development Wiki
Jump to: navigation, search
m (Reverted edits by Indiekiduk (talk) to last revision by Macmade)
("Development" vs "Release" caches)
 
(26 intermediate revisions by 8 users not shown)
Line 5: Line 5:
 
If you're looking for binaries or libraries inside of <tt>[http://theiphonewiki.com/wiki//System/Library/Frameworks /System/Library/Frameworks]</tt> or <tt>/System/Library/PrivateFrameworks</tt> (or other directories) and can't, this is why.
 
If you're looking for binaries or libraries inside of <tt>[http://theiphonewiki.com/wiki//System/Library/Frameworks /System/Library/Frameworks]</tt> or <tt>/System/Library/PrivateFrameworks</tt> (or other directories) and can't, this is why.
  
OS X also uses a shared cache. Unlike iOS, OS X ships with the source binaries still on-disk, particularly so it can be updated with [https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/update_dyld_shared_cache.1.html update_dyld_shared_cache]. The cache is only vaguely documented in dyld man pages.
+
As of iOS 13.5 application code is now in frameworks in the shared cache. The binaries you see in <tt>/Applications</tt> or <tt>/private/var/staged_system_apps</tt> are now just shims, so if you attempt to class-dump them it will error out as no ObjC section will be found.
  
= Cache location =
+
OS X, along with any other *OS released by apple also uses a shared cache. Unlike iOS, OS X ships with the source binaries still on-disk, particularly so it can be updated with [https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/update_dyld_shared_cache.1.html update_dyld_shared_cache]. The cache is only vaguely documented in dyld man pages.
 +
 
 +
 
 +
== Extracting Frameworks and Libraries. ==
 +
 
 +
Since iOS 8, the SDK no longer includes extracted frameworks. The only way to obtain the libraries running on your device is via extraction.
 +
 
 +
Extracting the shared cache is useful in a few situations:
 +
 
 +
* Linking against a framework or library not available in the public SDK
 +
* Using class-dump or similar tools to analyze Private Frameworks.
 +
 
 +
Until a tool capable of fixing extracted binaries is released, or unless you're able to obtain a .development cache, extracted framework may not be worth attempting to use in Reverse Engineering.
 +
 
 +
=== Obtaining a shared cache ===
 +
 
 +
The easiest method for getting a dyld_shared_cache is as such:
 +
 
 +
# Download or locate an ipsw for the target version and device 
 +
# Rename it to a .zip, extract, and locate the largest .dmg 
 +
# Mount the dmg and navigate to /System/Library/Caches/com.apple.dyld/ 
 +
# Copy the dyld_shared_cache to your machine. 
 +
 
 +
==== Other methods for cache retrieval ====
 +
 
 +
Since ASLR was implemented in iOS, trivial ways to pull the cache off the device have provided a "broken" cache, which can't be processed correctly by the aforementioned tools. This is because when read by processes in which ASLR is enabled, some offsetting is applied to the cache too. In order to circumvent this issue and pull a "valid" shared cache off the device, there are different options:
 +
 
 +
* Copy the cache off the device using a program on which ASLR has been explicitly disabled, using the <tt>-mdynamic-no-pic</tt> compile flag.
 +
* Read the cache explicitly from the filesystem by setting the <tt>F_NOCACHE</tt> flag on the cache's file descriptor.
 +
* Copy the cache through AFC (filesystem browsers which use an AFC connection are fine) - on iOS 7 and 8, you'll want to install the package ''Apple File Conduit "2"'', hosted/maintained by saurik.
 +
* Use the copy that is probably laying around on your computer in <tt>~/Library/Developer/Xcode/iOS\ DeviceSupport/</tt> if you have Xcode.
 +
 
 +
Alternatively, [https://github.com/npupyshev/dt.fetchsymbols dt.fetchsymbols] can be used to extract the cache from an iOS device. This tool doesn't require file system access (jailbreak) or app installation.
 +
 
 +
== Shared cache static analysis ==
 +
 
 +
Analyzing Private Frameworks can provide powerful insight into how iOS works, and can assist with writing better code as an extension developer.
 +
 
 +
If you're not sure what you need, and/or you don't have time to waste, you likely want Simulator Runtime frameworks.
 +
 
 +
=== Simulator Runtime Frameworks ===
 +
 
 +
The simulator runtime generated by XCode includes fully symbolicated, perfect x64 binaries for Private and Public frameworks. For most shared_cache analysis, unless you own IDA 7.5, you likely will be fine with using these. They're located in ''Add the folder here''
 +
 
 +
=== XCode's Automatic Extraction ===
 +
 
 +
Before you go through the painful extraction process, macOS users should check `~/Library/Developer/Xcode/iOS DeviceSupport`. When you build and deploy a project via XCode, it will automatically extract the entire cache to this directory. This eliminates the need to deal with other tools.
 +
 
 +
=== Disassemblers capable of extraction ===
 +
 
 +
Owning IDA 7.5 is one of the few reasons to bother using a real cache for Reverse Engineering private frameworks. Other tools are not capable of doing so easily if at all.
 +
 
 +
==== IDA Pro ====
 +
 
 +
Common illegally distributed copies of IDA don't include these utilities, and you won't be able to use features documented here or elsewhere on a stolen copy.
 +
 
 +
IDA 7.5 includes a plethora of tools that make working with the dsc much more tolerable. It entirely eliminates the need for extraction or third-party scripts to fix output.
 +
 
 +
Details and instructions on using the integrated tools can be found on the [[IDA Pro]] page. If you own a copy, check the IDA page on this wiki for a guide on using it.
 +
 
 +
==== Hopper ====
 +
 
 +
Hopper is capable of loading singular modules from the shared cache. It does not fix references and as such produces mostly useless output.
 +
 
 +
It also allows loading the entire shared cache, but tends to crash when loading it
 +
 
 +
==== Ghidra ====
 +
 
 +
Ghidra is capable of loading the shared cache
 +
 
 +
''More information is needed here''
 +
 
 +
=== Standalone tools ===
 +
 
 +
Working on current iOS versions:
 +
 
 +
* [http://opensource.apple.com/source/dyld/ dsc_extractor (source code)]. More info [https://gist.github.com/NSExceptional/85527151eeec4b0640187a0a165da1cd here]. It produces the best results among all tools, but without branch islands workaround. Do note this is the exact same output provided by XCode automatically.
 +
 
 +
Tools for previous iOS versions:
 +
 
 +
* [https://github.com/kennytm/Miscellaneous/downloads dyld_decache] by KennyTM~ to extract these dylibs.
 +
* [https://gist.github.com/455086/ DySlim] by comex to mount the whole cache file on Mac OS X.
 +
* [https://github.com/phoenix3200/decache decache] by phoenixdev to nearly perfectly extract dylibs from iOS <= 6 cache file.
 +
* [http://www.newosxbook.com/index.php?page=downloads jtool] is another option starting from iOS 8.
 +
* [https://github.com/comex/imaon2 yasce] by comex is/was the best option for iOS 8 (and above), but you will need a nightly build version of rust; something like "rustc 1.9.0-nightly (339a409bf 2016-03-01)".
 +
* [https://github.com/macmade/dyld_cache_extract dyld_cache_extract] by macmade that works on macOS and provides a complete GUI. Clone repo and do 'git submodule update --remote' before buidling. It was reported to be not working on iOS 10.2's dyld_shared_cache_armv7s; gave a 561.1MB executable file.
 +
 
 +
 
 +
== Cache location ==
  
 
The cache is located in <tt>/System/Library/Caches/com.apple.dyld/dyld_shared_cache_armX</tt>, where X can be:
 
The cache is located in <tt>/System/Library/Caches/com.apple.dyld/dyld_shared_cache_armX</tt>, where X can be:
Line 28: Line 116:
 
| 64
 
| 64
 
| ARMv8
 
| ARMv8
 +
|-
 +
| 64e
 +
| ARMv8.3
 
|}
 
|}
  
= Cache extraction =
+
== "Development" vs "Release" caches ==
 +
 
 +
The information here is purely educational, and while curiosity is encouraged, avoid giving legitimacy to Apple's smears on jailbreak developers.
 +
 
 +
=== Creation/background ===
  
It used to be possible to compile and link from an iOS device without an SDK, but the introduction of dyld_shared_cache has made using an SDK a necessity. Alternatively, one may need to extract the appropriate dylibs from the dyld_shared_cache, if it's not available in the SDK.
+
The source code for the tool used at apple is here:
  
Starting with iOS 8, the framework binaries shipped with the iOS SDK only contain the symbols, not the compiled code anymore. Binaries extracted from the dyld_shared_cache contain all compiled code and are therefore useful for reverse-engineering purposes.
+
https://opensource.apple.com/source/dyld/dyld-733.6/dyld3/shared-cache/CacheBuilder.cpp.auto.html
  
Options:
+
Specifically linked is the file responsible for determining if a development build is being created, and if so to avoid removing stubs. This is likely done to make debugging issues in frameworks much less painful.
  
* You could use [https://github.com/kennytm/Miscellaneous/downloads dyld_decache] by KennyTM~ to extract these dylibs.
+
Evidence of these builds is present in dyld source (''link to a github mirror here please''), and they're likely used in internal iOS builds.
* Alternatively, you could use [https://gist.github.com/455086/ DySlim] by comex to mount the whole cache file on Mac OS X.
 
* [https://github.com/phoenix3200/decache decache] by phoenixdev also works quite well. Out of the tools presented here, it currently produces the most usable results.
 
* [http://opensource.apple.com/source/dyld/ dsc_extractor (source code)]. More info [http://lightbulbone.tumblr.com/post/56546834100/ios-shared-cache-extraction here].
 
* [http://www.newosxbook.com/index.php?page=downloads jtool] is another option if other tools fail (which seems to be common starting with iOS 8).
 
* [https://github.com/comex/imaon2 yasce] by comex seems to currently provide the best output for iOS8+ but have fun getting the right version of rust running.  You probably want something like "rustc 1.9.0-nightly (339a409bf 2016-03-01)".
 
* [https://github.com/macmade/dyld_cache_extract dyld_cache_extract] by macmade. Works on macOS and provides a complete GUI.
 
  
== Example usage for decache ==
+
 
 +
=== IDA and .development caches ===
 +
 
 +
Short answer: Not worth your time in recent versions.
 +
 
 +
IDA Offers tools to deal with the lack of stubs in release caches. Using it with the most recent build of IDA is pointless due to the dscu provided.
 +
 
 +
Current versions have shortcomings regarding recognizing certain segments and omitting information, but DSCU can be used to load in that information.
 +
 
 +
 
 +
=== Other Disassemblers ===
 +
 
 +
''Information on how other disassemblers interact with these caches is needed.''
 +
 
 +
== Example usage for dsc_extractor ==
 +
 
 +
''Given that this is a bit of a source-code dump of an apple tool, it probably should be moved to its own page''
 +
 
 +
Sorry this is still work in progress as of iOS 13.4.1 - the binary output is not usable in either Hopper or class-dump.
 +
 
 +
This tool is different in that it dumps every binary in the cache compared to other tools that allow extraction of individual binaries. We need to create a c++ file and use Apple's test program code to build an executable:
 +
 
 +
<source lang=bash>
 +
cd Downloads
 +
mkdir dsc_extractor
 +
cd dsc_extractor
 +
mate dsc_extractor.cpp
 +
</source>
 +
 
 +
Copy and paste in the following Apple test program taken from the dyld package (look after the code for instructions on how to download it yourself).
 +
 
 +
<source lang=c++>
 +
// test program
 +
#include <stdio.h>
 +
#include <stddef.h>
 +
#include <dlfcn.h>
 +
 
 +
 
 +
typedef int (*extractor_proc)(const char* shared_cache_file_path, const char* extraction_root_path,
 +
                              void (^progress)(unsigned current, unsigned total));
 +
 
 +
int main(int argc, const char* argv[])
 +
{
 +
    if ( argc != 3 ) {
 +
        fprintf(stderr, "usage: dsc_extractor <path-to-cache-file> <path-to-device-dir>\n");
 +
        return 1;
 +
    }
 +
 
 +
    //void* handle = dlopen("/Volumes/my/src/dyld/build/Debug/dsc_extractor.bundle", RTLD_LAZY);
 +
    void* handle = dlopen("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/usr/lib/dsc_extractor.bundle", RTLD_LAZY);
 +
    if ( handle == NULL ) {
 +
        fprintf(stderr, "dsc_extractor.bundle could not be loaded\n");
 +
        return 1;
 +
    }
 +
 
 +
    extractor_proc proc = (extractor_proc)dlsym(handle, "dyld_shared_cache_extract_dylibs_progress");
 +
    if ( proc == NULL ) {
 +
        fprintf(stderr, "dsc_extractor.bundle did not have dyld_shared_cache_extract_dylibs_progress symbol\n");
 +
        return 1;
 +
    }
 +
 
 +
    int result = (*proc)(argv[1], argv[2], ^(unsigned c, unsigned total) { printf("%d/%d\n", c, total); } );
 +
    fprintf(stderr, "dyld_shared_cache_extract_dylibs_progress() => %d\n", result);
 +
    return 0;
 +
}
 +
</source>
 +
 
 +
This test program was taken from dydl 733.6. Get latest by browsing the versions [https://opensource.apple.com/source/dyld/ here] and download by swapping in a version number like:
 +
<source lang=bash>
 +
wget http://opensource.apple.com/tarballs/dyld/dyld-733.6.tar.gz
 +
tar xvf dyld-733.6.tar.gz
 +
cd dyld-733.6/launch_cache
 +
mate dyld-733.6/launch_cache/dsc_extractor.cpp
 +
</source>
 +
 
 +
Scroll to the bottom to find the test program code in the big comment.
 +
 
 +
Back to what we were doing, to build the file we created and install:
 +
 
 +
<source lang=bash>
 +
clang++ -o dsc_extractor dsc_extractor.cpp
 +
cp dsc_extractor ~/bin/
 +
chmod u+x ~/bin/dsc_extractor
 +
</source>
 +
 
 +
To use (Change the iOS version to what is in your folder, Xcode will copy the cache file from a connected device):
 +
<source lang=bash>
 +
cd ~/Library/Developer/Xcode/iOS DeviceSupport/13.4.1 (17E262)
 +
dsc_extractor Symbols/System/Library/Caches/com.apple.dyld/dyld_shared_cache_arm64 Binaries
 +
</source>
 +
 
 +
After a brief pause you'll see a lot of output as it dumps each binary, e.g.
 +
<source lang=bash>
 +
0/1805
 +
1/1805
 +
2/1805
 +
3/1805
 +
4/1805
 +
5/1805
 +
</source>
 +
 
 +
And now, you can find all of the dumped binaries in the Binaries folder, the problem is the binaries aren't usable in class-dump and errors with "Cannot find offset", e.g.:
 +
 
 +
<source lang=bash>
 +
class-dump -H Binaries/System/Library/PrivateFrameworks/WiFiKit.framework/WiFiKit
 +
2020-05-02 11:58:35.887 class-dump[47492:2792183] Error: Cannot find offset for address 0x201c9409718 in dataOffsetForAddress:
 +
</source>
 +
 
 +
=== Example usage for decache ===
  
 
This will extract the binary of the private framework SpringBoardServices
 
This will extract the binary of the private framework SpringBoardServices
Line 56: Line 253:
 
If you get a message about an unsupported load command, ignore it. decache does not support some newer mach-o load commands, so the binary won't be able to run probably in the most cases. But for linking or reverse-engineering purposes it is still usable.
 
If you get a message about an unsupported load command, ignore it. decache does not support some newer mach-o load commands, so the binary won't be able to run probably in the most cases. But for linking or reverse-engineering purposes it is still usable.
  
== Example usage for jtool ==
+
=== Example usage for jtool ===
  
 
To extract a specific binary from the cache ("UIKit" can be replaced with a different framework or library):
 
To extract a specific binary from the cache ("UIKit" can be replaced with a different framework or library):
Line 65: Line 262:
  
 
<source lang=bash>
 
<source lang=bash>
jtool -lv cache_armv7 | cut -c 24- | tail +5 | while read line ; do jtool -extract $line cache_armv7 ; done
+
cache=dyld_shared_cache_arm64
 +
mkdir -p extracted && jtool -lv $cache | cut -c 24- | tail +5 | while read line; do mkdir -p extracted/"$(dirname "$line")"; jtool -extract $line $cache; mv $cache."$(basename "$line")" extracted/$line; done
 
</source>
 
</source>
  
=== Problems with jtool ===
+
==== Problems with jtool ====
  
 
Please be aware that decache produces currently (16.04.15) better and more usable results then jtool, as jtool fails to resolve and fix the "uniqued" objectiv c selectors correctly.
 
Please be aware that decache produces currently (16.04.15) better and more usable results then jtool, as jtool fails to resolve and fix the "uniqued" objectiv c selectors correctly.
Line 74: Line 272:
 
Apple "uniques" objectiv c selectors, such as "alloc" (alloc is used almost everywhere), which are used in more then one place, into a single one. When extracting an image from the cache, the address of such a shared selector will most likely not be in the extracted image anymore, so this needs to fixed, which jtool apparently fails to do. (For more information: http://opensource.apple.com/source/dyld/dyld-132.13/launch-cache/update_dyld_shared_cache.cpp, look at the class ObjCSelectorUniquer)
 
Apple "uniques" objectiv c selectors, such as "alloc" (alloc is used almost everywhere), which are used in more then one place, into a single one. When extracting an image from the cache, the address of such a shared selector will most likely not be in the extracted image anymore, so this needs to fixed, which jtool apparently fails to do. (For more information: http://opensource.apple.com/source/dyld/dyld-132.13/launch-cache/update_dyld_shared_cache.cpp, look at the class ObjCSelectorUniquer)
  
= Cache retrieval =
+
==== Not working since iOS 11 ====
  
Since ASLR was implemented in iOS, trivial ways to pull the cache off the device have provided a "broken" cache, which can't be processed correctly by the aforementioned tools. This is because when read by processes in which ASLR is enabled, some offsetting is applied to the cache too. In order to circumvent this issue and pull a "valid" shared cache off the device, there are different options:
+
jtool2 is the newer version this user reports it not working on the iOS 11 shared cache - shows a warning "File is likely truncated (or header corrupt?)" and then doesn't get past "LC_DYLD_INFO..." http://newosxbook.com/forum/viewtopic.php?f=3&t=19577&start=50#p24183 Still same error exists on the cache from iOS 13.
  
* Copy the cache off the device using a program on which ASLR has been explicitly disabled, using the <tt>-mdynamic-no-pic</tt> compile flag.
+
<source lang=bash>
* Read the cache explicitly from the filesystem by setting the <tt>F_NOCACHE</tt> flag on the cache's file descriptor.
+
$ ./jtool2 -e Stocks ./dyld_shared_cache_arm64
* Copy the cache through AFC (filesystem browsers which use an AFC connection are fine) - on iOS 7 and 8, you'll want to install the package ''Apple File Conduit "2"'', hosted/maintained by saurik.
+
Warning: File is likely truncated (or header corrupt?) Binding opcodes falls outside file
* Pull the cache off a decrypted root filesystem DMG which you can find inside the IPSW.
+
Warning: File is likely truncated (or header corrupt?) Binding opcodes falls outside file
* Use the copy that is probably laying around on your computer in "~/Library/Developer/Xcode/iOS DeviceSupport/" if you have Xcode.
+
Warning: File is likely truncated (or header corrupt?) LC_FUNCTION_STARTS falls outside file
 
+
0x176d9000-0x17731000 __TEXT  (360448 bytes)
Alternatively, [https://github.com/npupyshev/dt.fetchsymbols dt.fetchsymbols] can be used to extract the cache from an iOS device. This tool doesn't require file system access (jailbreak) or app installation.
+
0x2ccc2070-0x2ccd54a8 __DATA_CONST  (78904 bytes)
 +
0x30571f60-0x30575f60 __DATA  (16384 bytes)
 +
0x314c9a28-0x314ca000 __DATA_DIRTY  (1496 bytes)
 +
0x316b3000-0x37064000 __LINKEDIT  (94048256 bytes)
 +
  0x31e3c4d0-0x31e3eaf8 Exports  (9768 bytes)
 +
  0x34689f18-0x346912f8 Symbol Table  (29664 bytes)
 +
  0x35a46500-0x35a46b90 Function Starts  (1680 bytes)
 +
  0x35bb1930-0x35bb1960 Data In Code  (48 bytes)
 +
  0x35d66710-0x37029aba String Table  (19674026 bytes)
 +
LC_DYLD_INFO...
 +
</source>
  
= Class dumping =
+
== Class dumping ==
  
 
See [[Reverse_Engineering_Tools#class-dump.2C_class_dump_z.2C_classdump-dyld|this section of Reverse Engineering Tools]].
 
See [[Reverse_Engineering_Tools#class-dump.2C_class_dump_z.2C_classdump-dyld|this section of Reverse Engineering Tools]].
 
   
 
   
= External Links =
+
== External Links ==
  
 
* [http://blog.howett.net/?p=75 Cache or Check?] — an analysis of the dyld_shared_cache system by D. Howett.
 
* [http://blog.howett.net/?p=75 Cache or Check?] — an analysis of the dyld_shared_cache system by D. Howett.
 +
* [https://github.com/deepinstinct/dsc_fix dsc_fix] — an IDA script that aids in reverse engineering dyld_shared_cache libraries

Latest revision as of 05:01, 19 June 2020


Since iPhone OS 3.1, all system (private and public) libraries have been combined into a big cache file to improve performance. The original files are redundant and thus eliminated from the system.

If you're looking for binaries or libraries inside of /System/Library/Frameworks or /System/Library/PrivateFrameworks (or other directories) and can't, this is why.

As of iOS 13.5 application code is now in frameworks in the shared cache. The binaries you see in /Applications or /private/var/staged_system_apps are now just shims, so if you attempt to class-dump them it will error out as no ObjC section will be found.

OS X, along with any other *OS released by apple also uses a shared cache. Unlike iOS, OS X ships with the source binaries still on-disk, particularly so it can be updated with update_dyld_shared_cache. The cache is only vaguely documented in dyld man pages.


Extracting Frameworks and Libraries.

Since iOS 8, the SDK no longer includes extracted frameworks. The only way to obtain the libraries running on your device is via extraction.

Extracting the shared cache is useful in a few situations:

  • Linking against a framework or library not available in the public SDK
  • Using class-dump or similar tools to analyze Private Frameworks.

Until a tool capable of fixing extracted binaries is released, or unless you're able to obtain a .development cache, extracted framework may not be worth attempting to use in Reverse Engineering.

Obtaining a shared cache

The easiest method for getting a dyld_shared_cache is as such:

  1. Download or locate an ipsw for the target version and device
  2. Rename it to a .zip, extract, and locate the largest .dmg
  3. Mount the dmg and navigate to /System/Library/Caches/com.apple.dyld/
  4. Copy the dyld_shared_cache to your machine.

Other methods for cache retrieval

Since ASLR was implemented in iOS, trivial ways to pull the cache off the device have provided a "broken" cache, which can't be processed correctly by the aforementioned tools. This is because when read by processes in which ASLR is enabled, some offsetting is applied to the cache too. In order to circumvent this issue and pull a "valid" shared cache off the device, there are different options:

  • Copy the cache off the device using a program on which ASLR has been explicitly disabled, using the -mdynamic-no-pic compile flag.
  • Read the cache explicitly from the filesystem by setting the F_NOCACHE flag on the cache's file descriptor.
  • Copy the cache through AFC (filesystem browsers which use an AFC connection are fine) - on iOS 7 and 8, you'll want to install the package Apple File Conduit "2", hosted/maintained by saurik.
  • Use the copy that is probably laying around on your computer in ~/Library/Developer/Xcode/iOS\ DeviceSupport/ if you have Xcode.

Alternatively, dt.fetchsymbols can be used to extract the cache from an iOS device. This tool doesn't require file system access (jailbreak) or app installation.

Shared cache static analysis

Analyzing Private Frameworks can provide powerful insight into how iOS works, and can assist with writing better code as an extension developer.

If you're not sure what you need, and/or you don't have time to waste, you likely want Simulator Runtime frameworks.

Simulator Runtime Frameworks

The simulator runtime generated by XCode includes fully symbolicated, perfect x64 binaries for Private and Public frameworks. For most shared_cache analysis, unless you own IDA 7.5, you likely will be fine with using these. They're located in Add the folder here

XCode's Automatic Extraction

Before you go through the painful extraction process, macOS users should check `~/Library/Developer/Xcode/iOS DeviceSupport`. When you build and deploy a project via XCode, it will automatically extract the entire cache to this directory. This eliminates the need to deal with other tools.

Disassemblers capable of extraction

Owning IDA 7.5 is one of the few reasons to bother using a real cache for Reverse Engineering private frameworks. Other tools are not capable of doing so easily if at all.

IDA Pro

Common illegally distributed copies of IDA don't include these utilities, and you won't be able to use features documented here or elsewhere on a stolen copy.

IDA 7.5 includes a plethora of tools that make working with the dsc much more tolerable. It entirely eliminates the need for extraction or third-party scripts to fix output.

Details and instructions on using the integrated tools can be found on the IDA Pro page. If you own a copy, check the IDA page on this wiki for a guide on using it.

Hopper

Hopper is capable of loading singular modules from the shared cache. It does not fix references and as such produces mostly useless output.

It also allows loading the entire shared cache, but tends to crash when loading it

Ghidra

Ghidra is capable of loading the shared cache

More information is needed here

Standalone tools

Working on current iOS versions:

  • dsc_extractor (source code). More info here. It produces the best results among all tools, but without branch islands workaround. Do note this is the exact same output provided by XCode automatically.

Tools for previous iOS versions:

  • dyld_decache by KennyTM~ to extract these dylibs.
  • DySlim by comex to mount the whole cache file on Mac OS X.
  • decache by phoenixdev to nearly perfectly extract dylibs from iOS <= 6 cache file.
  • jtool is another option starting from iOS 8.
  • yasce by comex is/was the best option for iOS 8 (and above), but you will need a nightly build version of rust; something like "rustc 1.9.0-nightly (339a409bf 2016-03-01)".
  • dyld_cache_extract by macmade that works on macOS and provides a complete GUI. Clone repo and do 'git submodule update --remote' before buidling. It was reported to be not working on iOS 10.2's dyld_shared_cache_armv7s; gave a 561.1MB executable file.


Cache location

The cache is located in /System/Library/Caches/com.apple.dyld/dyld_shared_cache_armX, where X can be:

X Device ARM Architecture
v6 ARMv6
v7 ARMv7
v7s
v7k
64 ARMv8
64e ARMv8.3

"Development" vs "Release" caches

The information here is purely educational, and while curiosity is encouraged, avoid giving legitimacy to Apple's smears on jailbreak developers.

Creation/background

The source code for the tool used at apple is here:

https://opensource.apple.com/source/dyld/dyld-733.6/dyld3/shared-cache/CacheBuilder.cpp.auto.html

Specifically linked is the file responsible for determining if a development build is being created, and if so to avoid removing stubs. This is likely done to make debugging issues in frameworks much less painful.

Evidence of these builds is present in dyld source (link to a github mirror here please), and they're likely used in internal iOS builds.


IDA and .development caches

Short answer: Not worth your time in recent versions.

IDA Offers tools to deal with the lack of stubs in release caches. Using it with the most recent build of IDA is pointless due to the dscu provided.

Current versions have shortcomings regarding recognizing certain segments and omitting information, but DSCU can be used to load in that information.


Other Disassemblers

Information on how other disassemblers interact with these caches is needed.

Example usage for dsc_extractor

Given that this is a bit of a source-code dump of an apple tool, it probably should be moved to its own page

Sorry this is still work in progress as of iOS 13.4.1 - the binary output is not usable in either Hopper or class-dump.

This tool is different in that it dumps every binary in the cache compared to other tools that allow extraction of individual binaries. We need to create a c++ file and use Apple's test program code to build an executable:

cd Downloads 
mkdir dsc_extractor
cd dsc_extractor
mate dsc_extractor.cpp

Copy and paste in the following Apple test program taken from the dyld package (look after the code for instructions on how to download it yourself).

// test program
#include <stdio.h>
#include <stddef.h>
#include <dlfcn.h>


typedef int (*extractor_proc)(const char* shared_cache_file_path, const char* extraction_root_path,
                              void (^progress)(unsigned current, unsigned total));

int main(int argc, const char* argv[])
{
    if ( argc != 3 ) {
        fprintf(stderr, "usage: dsc_extractor <path-to-cache-file> <path-to-device-dir>\n");
        return 1;
    }

    //void* handle = dlopen("/Volumes/my/src/dyld/build/Debug/dsc_extractor.bundle", RTLD_LAZY);
    void* handle = dlopen("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/usr/lib/dsc_extractor.bundle", RTLD_LAZY);
    if ( handle == NULL ) {
        fprintf(stderr, "dsc_extractor.bundle could not be loaded\n");
        return 1;
    }

    extractor_proc proc = (extractor_proc)dlsym(handle, "dyld_shared_cache_extract_dylibs_progress");
    if ( proc == NULL ) {
        fprintf(stderr, "dsc_extractor.bundle did not have dyld_shared_cache_extract_dylibs_progress symbol\n");
        return 1;
    }

    int result = (*proc)(argv[1], argv[2], ^(unsigned c, unsigned total) { printf("%d/%d\n", c, total); } );
    fprintf(stderr, "dyld_shared_cache_extract_dylibs_progress() => %d\n", result);
    return 0;
}

This test program was taken from dydl 733.6. Get latest by browsing the versions here and download by swapping in a version number like:

wget http://opensource.apple.com/tarballs/dyld/dyld-733.6.tar.gz
tar xvf dyld-733.6.tar.gz
cd dyld-733.6/launch_cache
mate dyld-733.6/launch_cache/dsc_extractor.cpp

Scroll to the bottom to find the test program code in the big comment.

Back to what we were doing, to build the file we created and install:

clang++ -o dsc_extractor dsc_extractor.cpp
cp dsc_extractor ~/bin/
chmod u+x ~/bin/dsc_extractor

To use (Change the iOS version to what is in your folder, Xcode will copy the cache file from a connected device):

cd ~/Library/Developer/Xcode/iOS DeviceSupport/13.4.1 (17E262)
dsc_extractor Symbols/System/Library/Caches/com.apple.dyld/dyld_shared_cache_arm64 Binaries

After a brief pause you'll see a lot of output as it dumps each binary, e.g.

0/1805
1/1805
2/1805
3/1805
4/1805
5/1805

And now, you can find all of the dumped binaries in the Binaries folder, the problem is the binaries aren't usable in class-dump and errors with "Cannot find offset", e.g.:

class-dump -H Binaries/System/Library/PrivateFrameworks/WiFiKit.framework/WiFiKit 
2020-05-02 11:58:35.887 class-dump[47492:2792183] Error: Cannot find offset for address 0x201c9409718 in dataOffsetForAddress:

Example usage for decache

This will extract the binary of the private framework SpringBoardServices

decache -c path/to/dyld_shared_cache -x /System/Library/PrivateFrameworks/SpringBoardServices.framework/SpringBoardServices -o SpringBoardServices

If you get a message about an unsupported load command, ignore it. decache does not support some newer mach-o load commands, so the binary won't be able to run probably in the most cases. But for linking or reverse-engineering purposes it is still usable.

Example usage for jtool

To extract a specific binary from the cache ("UIKit" can be replaced with a different framework or library):

jtool -extract UIKit path/to/dyld_shared_cache

An example of one way to dump all the binaries at once (be careful with this, it creates huge files):

cache=dyld_shared_cache_arm64
mkdir -p extracted && jtool -lv $cache | cut -c 24- | tail +5 | while read line; do mkdir -p extracted/"$(dirname "$line")"; jtool -extract $line $cache; mv $cache."$(basename "$line")" extracted/$line; done

Problems with jtool

Please be aware that decache produces currently (16.04.15) better and more usable results then jtool, as jtool fails to resolve and fix the "uniqued" objectiv c selectors correctly.

Apple "uniques" objectiv c selectors, such as "alloc" (alloc is used almost everywhere), which are used in more then one place, into a single one. When extracting an image from the cache, the address of such a shared selector will most likely not be in the extracted image anymore, so this needs to fixed, which jtool apparently fails to do. (For more information: http://opensource.apple.com/source/dyld/dyld-132.13/launch-cache/update_dyld_shared_cache.cpp, look at the class ObjCSelectorUniquer)

Not working since iOS 11

jtool2 is the newer version this user reports it not working on the iOS 11 shared cache - shows a warning "File is likely truncated (or header corrupt?)" and then doesn't get past "LC_DYLD_INFO..." http://newosxbook.com/forum/viewtopic.php?f=3&t=19577&start=50#p24183 Still same error exists on the cache from iOS 13.

$ ./jtool2 -e Stocks ./dyld_shared_cache_arm64 
Warning: File is likely truncated (or header corrupt?) Binding opcodes falls outside file
Warning: File is likely truncated (or header corrupt?) Binding opcodes falls outside file
Warning: File is likely truncated (or header corrupt?) LC_FUNCTION_STARTS falls outside file
0x176d9000-0x17731000 __TEXT   (360448 bytes)
0x2ccc2070-0x2ccd54a8 __DATA_CONST   (78904 bytes)
0x30571f60-0x30575f60 __DATA   (16384 bytes)
0x314c9a28-0x314ca000 __DATA_DIRTY   (1496 bytes)
0x316b3000-0x37064000 __LINKEDIT   (94048256 bytes)
   0x31e3c4d0-0x31e3eaf8 Exports   (9768 bytes)
   0x34689f18-0x346912f8 Symbol Table   (29664 bytes)
   0x35a46500-0x35a46b90 Function Starts   (1680 bytes)
   0x35bb1930-0x35bb1960 Data In Code   (48 bytes)
   0x35d66710-0x37029aba String Table   (19674026 bytes)
LC_DYLD_INFO...

Class dumping

See this section of Reverse Engineering Tools.

External Links

  • Cache or Check? — an analysis of the dyld_shared_cache system by D. Howett.
  • dsc_fix — an IDA script that aids in reverse engineering dyld_shared_cache libraries