Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup after Link2SD/Apps2SD #674

Closed
Yowlen opened this issue Jan 4, 2017 · 31 comments
Closed

Cleanup after Link2SD/Apps2SD #674

Yowlen opened this issue Jan 4, 2017 · 31 comments

Comments

@Yowlen
Copy link

Yowlen commented Jan 4, 2017

Link2SD is an app that automates the various App2SD scripts found on various Android root forums. It'd be nice to have a detection system for app remnants on the primary & secondary SD card partitions. It mounts the secondary SD card partition at /data/sdext2, and stores things as follows:
/data/app/app.package.dir > /data/sdext2/app.package.dir
/data/data/app.package.dir > /data/sdext2/data/app.package.dir
/data/dalvik-cache > /data/sdext2/dalvik-cache
/storage/emulated/0/Android/data > /data/sdext2/Link2SD/bind/data OR /storage/sdcard1/Link2SD/bind/data
/storage/emulated/0/Android/obb > /data/sdext2/Link2SD/bind/obb OR /storage/sdcard1/Link2SD/bind/obb

Edit: Forgot about the Dalvik cache. XD

@d4rken
Copy link
Member

d4rken commented Jan 4, 2017

Can you show me the directories contents you want SD Maid scan?

Doesn't Link2SD clean up after the apps it linked?

@Yowlen
Copy link
Author

Yowlen commented Jan 5, 2017

Normally, yes, Link2SD does clean up after itself when an app gets uninstalled, but certain circumstances will prevent that activation. Resetting the device, for example. And Link2SD has its own built-in cleaner that can be activated within the app. However, the built-in cleaner needs a lot of work, as it currently wants to clear the entire /data/sdext2 folder of all files - including any user-created ones while leaving the folders - including the data for the apps that are no longer installed.

Anyway, this request is just so it can be easily cleaned all in one place w/ the rest of the system.

As for what to clean, it's just scanning for apps/data that are no longer installed but with an extra base search path for each section, which I listed above. The only thing I didn't mention before because I didn't think of it would be that an exception would have to be made for the /data/app > /data/sdext2 base folder pair (the first one I listed above) to recognize folders that aren't apps. Since Link2SD uses the package name as the folder name, any folders that don't start with "com.", "org.", "net.", "de.", or any other common package naming scheme can be ignored. The other folder pairs don't need this check since they have their own subfolders.

As an example, here's a small screenshot of part if my /data/sdext2 folder:
tmp_12209-screenshot_2017-01-05-09-11-561924716089
The first folder follows the common naming scheme, so it would be checked against the installed package lists just as if it were in /data/app. The others don't, so they'd be ignored.

The directory structure inside each base path is the same, so for example, /data/dalvik-cache has each architecture in separate subfolders, so the ARM architecture would have its stuff stored under /data/dalvik-cache/arm/package.name.dex, and if it gets moved with Link2SD, it's at /data/sdext2/dalvik-cache/arm/package.name.dex.

@d4rken
Copy link
Member

d4rken commented Jan 5, 2017 via email

@Yowlen
Copy link
Author

Yowlen commented Jan 5, 2017

@Yowlen Yowlen closed this as completed Jan 5, 2017
@Yowlen Yowlen reopened this Jan 5, 2017
@Yowlen
Copy link
Author

Yowlen commented Jan 5, 2017

I hate hitting the wrong buttons. lol

Anyway, there ya go.

@Yowlen
Copy link
Author

Yowlen commented Jan 5, 2017

Most people wouldn't even have a secondary partition in the first place, let alone use it for anything other than App2SD stuff. I think just ignoring Link2SD's subfolders & any without a period, along with any non-apk files in the root folder (for 4.x compatibility) would be sufficient.

Also, if you need it, I can give a directory structure from my Kit Kat tablet to help ensure that compatibility.

@d4rken
Copy link
Member

d4rken commented Jan 5, 2017 via email

@Yowlen
Copy link
Author

Yowlen commented Jan 5, 2017 via email

@d4rken d4rken added this to the Next Tasks milestone Jan 7, 2017
@d4rken
Copy link
Member

d4rken commented Jan 8, 2017

Got Link2SD 4.0.13 setup on a [email protected].

On your log from earlier:

Where does the dir/data/sdext2/Link2SD/bind/ come from and why is it not on the log from 4.4.2?
Also, did you manually place the dir /data/sdext2/Politics there?

Edit:
Other example: /data/sdext2/Linux, I think you put those files there, if so, how? Shell? Computer? App?

@Yowlen
Copy link
Author

Yowlen commented Jan 8, 2017

/data/sdext2/Link2SD/bind
^ This isn't on the 4.4.2 log because it's for the emulated storage mount points (/storage/emulated/0/Android/(Both data & obb)), which aren't present for the apps I linked on the 4.4.2 device. If I had linked one of those on my tablet, the folder would've been created. And just to clarify, this folder also exists on the primary external SD card partition, as Link2SD allows the user to choose for these two options since it uses the mount-o link option while the internal /data uses ln -s, which isn't supported on FAT partitions.

Politics & Linux are user-created folders, yes. I use the Linux folder to store chroot images for Linux Deploy. All folders from my 5.x log that relate to Link2SD are listed in my first post. All the rest are of my own creation (usually using a root explorer app) for various purposes.

@d4rken
Copy link
Member

d4rken commented Jan 8, 2017

  • /data/data/app.package.dir > /data/sdext2/data/app.package.dir
    We can match this by packagename.

  • /storage/emulated/0/Android/data > /data/sdext2/Link2SD/bind/data OR /storage/sdcard1/Link2SD/bind/data
    We can also match this by packagename.

  • /data/app/app.package.dir > /data/sdext2/app.package.dir (5.0+)
    We could match by detecting the -1 extensions, then check if it contains a base.apk, if so we get the packagename from the base.apk and check if that is installed

  • /data/app/app-1.apk > /data/sdext2/app-1.apk (<5.0)
    Get all files ending in .apk. Obtain the packagename from the .apk, check if that is installed.

  • /data/dalvik-cache > /data/sdext2/dalvik-cache
    @Yowlen ideas?

I don't have /data/sdext2/data, do I need the paid version for that?

@d4rken
Copy link
Member

d4rken commented Jan 8, 2017

On a side note, does Apps2SD and Link2SD use the same structure?

@d4rken d4rken changed the title [Feature Request][CorpseFinder] Link2SD Support Cleanup after Link2SD/Apps2SD Jan 8, 2017
@Yowlen
Copy link
Author

Yowlen commented Jan 8, 2017

For /data/sdext2/dalvik-cache I'd assume matching the .dex file to the package name would work.

/data/data > /data/sdext2/data is a paid feature, yes.

Apps2SD isn't an app itself. It's a script designed to be run at boot either by init.d or similar. There's many such scripts out there, and they use different methods. Some mount the 2nd partition to /sdext2 instead of /data/sdext2, for example. The advanced nature of Apps2SD's requirements (script editing & stuff) means it's unlikely you'll be able to account for them all, and Link2SD is the only app that's still being updated that automates the script generation. I don't know of any other app that works for this on 5.x+.

@d4rken
Copy link
Member

d4rken commented Jan 8, 2017

Seems almost the same. Apps2SD uses:

  • /data/app/some.app-1 > /data/sdext2/apk/some.app-1
    Seems cleaner than what Link2SD does (placing it in the root dir).
  • /storage/emulated/0/Android/data/some.app > /data/sdext2/Android/data/some.app instead of /data/sdext2/Link2SD/bind/data

With both being so similar, we could probably make the filter cover both? I don't see the difference you mean. I installed both apps and they both create scripts and symlink app files.

Testing on my Moto [email protected], file structure seems the same as your 5.0.1 ROM. We got the app-lib difference on 4.4.X, now we still need to check for difference on 4.1, 4.2 and 4.3.

How is multi-user environment handled? Seems both apps symlinking style would break on multi-user environments because /data/sdext2/data would get packagename collisions.

@Yowlen
Copy link
Author

Yowlen commented Jan 8, 2017 via email

@d4rken
Copy link
Member

d4rken commented Jan 9, 2017

Public data

Link2SD

/storage/emulated/0/Android/data/some.app > /data/sdext2/Link2SD/bind/data/some.app
/storage/emulated/0/Android/data/some.app > /storage/sdcard1/Link2SD/bind/data/some.app

Apps2SD:

/storage/emulated/0/Android/data/some.app > /data/sdext2/Android/data/some.app
/storage/emulated/0/Android/data/some.app > /storage/sdcard1/Apps2SD/Android/data/some.app

Obb data

Link2SD

/storage/emulated/0/Android/obb/some.app > /data/sdext2/Link2SD/bind/obb/some.app
/storage/emulated/0/Android/obb/some.app > /storage/sdcard1/Link2SD/bind/obb/some.app

Apps2SD

/storage/emulated/0/Android/obb/some.app > /data/sdext2/Android/obb/some.app
/storage/emulated/0/Android/obb/some.app > /storage/sdcard1/Apps2SD/Android/obb/some.app

Dalvik-Cache

/data/dalvik-cache/profiles/some.app is ignored by both Link2SD and Apps2SD.

Link2SD

/data/dalvik-cache/arm/some.app-1.odex > /data/sdext2/dalvik-cache/arm/some.app-1.odex
/data/dalvik-cache/arm64/some.app-1.odex > /data/sdext2/dalvik-cache/arm64/some.app-1.odex

Apps2SD

/data/dalvik-cache/arm/some.app-1.odex > /data/sdext2/dalvik-cache/some.app-1.odex

Private data

Both

/data/data/app.package.dir > /data/sdext2/data/app.package.dir

APK data

Link2SD

/data/app/some.app-1 > /data/sdext2/some.app-1
/data/app/some.other.app-1.apk > /data/sdext2/some.other.app-1.apk

Apps2SD

/data/app/some.app-1 > /data/sdext2/apk/some.app-1
/data/app/some.other.app-1.apk > /data/sdext2/apk/some.other.app-1.apk

App library data

Link2SD

/data/app-lib/some.app > /data/sdext2/app-lib/some.app (Only on <5.0).

Apps2SD

/data/app/some.app/lib > /data/sdext2/app-lib/some.app
/data/app-lib/some.app > /data/sdext2/app-lib/some.app

@Yowlen
Copy link
Author

Yowlen commented Jan 9, 2017

Public Data & Obb: Apps2SD can place them on /storage/sdcard1 like Link2SD.
Your path for Apps2SD for these is incorrect as I stated above. Apps2SD mimics the folder layout: /storage/emulated/0/Android/data > /data/sdext2/Android/data
/storage/emulated/0/Android/obb > /data/sdext2/Android/obb
For the primary SD card partition, Apps2SD uses the following:
/storage/emulated/0/Android/data > /storage/sdcard1/Apps2SD/Android/data
/storage/emulated/0/Android/obb > /storage/sdcard1/Apps2SD/Android/obb

Dalvik Cache: The profiles folder isn't linked. It's just a bunch of 0-byte markers from what I see. And there's a slight difference between how Link2SD and Apps2SD stores these. Apps2SD doesn't separate by architecture:
Link2SD: > /data/sdext2/dalvik-cache/[architecture]
Apps2SD: > /data/sdext2/dalvik-cache

App Library: While Link2SD moves the libraries with the data on 5.x+, Apps2SD does separate the two, using the same link destination as Link2SD does for 4.x-. i.e. /data/data/some.app/lib > /data/sdext2/app-lib/some.app

I'm starting up my 4.4.2 tablet to test Apps2SD now. I'll edit this when I have the results.

Edit: I can confirm Apps2SD uses the same layout on 4.4.2 as it does on 5.1.1.

@d4rken
Copy link
Member

d4rken commented Jan 9, 2017

I've updated my post, please check again. Did we miss anything?

Should differentiate between APIs when checking? Probably not, as otherwise we can't detect corpses from older Android versions. This is a root only thing anyways so the user could have upgraded his ROM.

I think we can easily parse the packagename in every case, not sure if for apks we should parse the packagename or try to get it from the apk. Probably the apk because it's safer?

So this seems to viable, "just" needs a lot of regex love and unit tests would probably be good too.

@Yowlen
Copy link
Author

Yowlen commented Jan 9, 2017

Seems accurate except for one thing I forgot about until I double-checked my 4.4.2 directory structure dump: Link2SD's Dalvik Cache on 4.x is like Apps2SD's layout - it doesn't separate by architecture. /data/dalvik-cache/some.app.dex > /data/sdext2/dalvik-cache/some.app.dex

Oh, and one other thing: Link2SD doesn't allow linking system apps, so no .odex files, just .dex. I haven't tested Apps2SD on a system app, but it appears to have the ability to do it given it shows the option to link .odex files.

@Gitoffthelawn
Copy link

Your reference is exceptionally useful: #674 (comment)

You may also want to add XInternalSD: http:https://repo.xposed.info/module/com.pyler.xinternalsd

@d4rken
Copy link
Member

d4rken commented Jan 10, 2017

@Gitoffthelawn I'm not sure it's the same functionality.

@Gitoffthelawn
Copy link

Hmmm... I'm not sure either because I am not too familiar with Link2SD and Apps2SD.

The idea behind XInternalSD is to map internal storage to the external microSD card.

@Yowlen
Copy link
Author

Yowlen commented Jan 10, 2017

Yeah, it's definitely not the same. Link2SD/Apps2SD uses ln -s & mount -o bind Linux terminal commands to reroute specific folders so they essentially exist in both locations while only writing to one location. Meanwhile, XInternalSD changes where Android's API thinks the internal SD card is, which essentially makes the simulated internal SD card inaccessible, and therefore no new information is written there anyway. And because it's done at the API level, even SD Maid would be fooled by XInternalSD.

Besides, Link2SD & Apps2SD use a second, normally-invisible/nonexistent partition on the SD card, which is what we're attempting to clean here. XInternalSD only changes file storage to where it would normally be on the primary partition, so SD Maid would already clean any junk there.

@Gitoffthelawn
Copy link

@Yowlen Thank you for the details. Much appreciated! 👍

@d4rken d4rken modified the milestones: Next Tasks, v4.6.0 Jan 14, 2017
@ivanunderdog
Copy link

Thank you fellas ☺

@d4rken
Copy link
Member

d4rken commented Jan 29, 2017

Looking for testers for this. If anyone has time please send a quick mail to support.

@d4rken d4rken closed this as completed Jan 30, 2017
@d4rken
Copy link
Member

d4rken commented Feb 3, 2017

@Gitoffthelawn @Yowlen Had a chance to test the filter in v4.6.0?

@Yowlen
Copy link
Author

Yowlen commented Feb 3, 2017

I observed it was in action as it detected the empty Apps2SD folder on the external SD's primary partition, but forgot to test more thoroughly until I got this reminder.

I created a few false folders in each area, and it seemed to work almost perfect. The only issue is for Link2SD's apk layout. Unless the folder has -1 or another number at the end of its name, it doesn't get detected by the filter. I realize this is due to the off chance someone creates a user folder with a period in it, but I think if it starts with the generic package names (com., org., de., tk., etc.) it should be OK to delete, since they're usually not used at the start of user-created folders. And anyone advanced enough to have user-created folders on the 2nd partition would know how to add exclusions in SD Maid.

@d4rken
Copy link
Member

d4rken commented Feb 3, 2017

The only issue is for Link2SD's apk layout. Unless the folder has -1 or another number at the end of its name, it doesn't get detected by the filter.

Doesn't every Link2SD apk have a -1 or -2 etc.?

Can you show me some samples where it's named differently?

@Yowlen
Copy link
Author

Yowlen commented Feb 4, 2017

Honestly, I'm not entirely sure. I think I recall seeing some in the past, but I may have confused it with /data/data or something.

@d4rken
Copy link
Member

d4rken commented Feb 4, 2017

I think that may be the case, but should I be wrong and you spot such behavior, please make new bug ticket, ok?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants