AirPlay broken for tvOS 10.2+

[postlund]

From tvOS 10.2 and onwards, device verification is now mandatory. This breaks AirPlay streaming for this version (and later). I will look into this as soon as I have some time.

https://www.google.se/amp/appleinsider.com/articles/17/03/29/tvos-102-update-requires-airplay-hardware-verification-breaks-third-party-streaming-apps/amp/

[funtax]

Maybe this project could get some help by Beamer, AirParrot or DoubleTwist?
They seem to have managed the hardware verification.

[mar-schmidt]

any updates here? :)

[funtax]

Hey there,

feel free to use my Java-library as a template: https://github.com/funtax/AirPlayAuth

You can simply import the project eg. into "IntelliJ Community Edition" and run the example :)

I'm the developer of two audio AirPlay-apps and can confirm that once the pairing has been done, the ongoing communication with the AirPlay-receiver is done like before.

Also, there seems to be no reliable way to check if pairing is required, beside checking the mdns-data for "appletv" and "pk".
Then authentication can also be made on ATVs which have the authentication not enabled, so you can simply always do the pairing in case it's an AppleTV.

[postlund]

@Ronelius As @funtax mentioned, he has reversed engineered the process so it should be possible to implement. There are some quirks that I need to figure out regarding the SRP process, but I hope to be able to use pysrp for that. My biggest issue at the moment is time, but fixing this would be really great (even though I'm not affected by it yet)!

[funtax]

@postlund Yes, the SRP-part was the hardest part: It's based on SRP6a in general, with some custom modifications which you can see inside my repository. Follow the code and not the comments on the methods.. I'm not sure if they really reflect the method-body right now.

If you can't pass this step, you might compare both implementations. But the SRP-engine would required a little modification for this as SRP contains a random "secret" generator which generated different values in every run. So you'd have to disable the generator.

I'm pressing thumbs, I have also done this by reverse-engineering another program so it's definitely possible.

Ps. You can use any ATV by manually enabling "device verification" under its AirPlay-settings.

[postlund]

@funtax Yeah, I can imagine 😄 I'm gonna try your library, add some logging and extract "reference" values for all the steps so I more easily can verify that I'm on the right track. Most key exchange schemes uses nounces to mitigate replay attacks, I assume that is what you mean with "secret"?

Yeah, I found that too. It's good that it's available on the ATV3 as well.

I have a question for you as well. In your implementation you randomize a "client id" as part of your auth token. Is there a reason for that? Based on the wireshark dumps I have taken in the past, the MAC-address of the interface used for AirPlay communication was used for this. Using the MAC-address would remove the need for saving an additional identifier.

[funtax]

@postlund Yes, the random nounce is the "secret".
On native devices, the MAC is the client-id, but as my devices are Android-devices (where the MAC is not available), I just use a random identifier.
Most important is, that the client-id and the EdDSA-key do not change and are stored together for further use. So to make it simple, I generate an "authToken" which is the random client-id "@" serialized-EdDSA-keypair.

If you have access to the MAC, just is this instead of the random ID.
But, I'm not sure what will happens if the user uses your library with EdDSA-key A+MAC, and eg. iTunes with EdDSA-key B+MAC, maybe the user has to re-authenticate after using the other software.

For my two apps, I have simply created one "authToken" and placed them in both applications for all users.

[postlund]

@funtax Ok, great! Then I know 😄

Ah, that makes sense. I see that it potentially could be a problem, yes. Then I might use an approach similar to you as well! Thanks for the hints, I will probably give you a mention if I hit any problems along the way 👍

[funtax]

@postlund I'm looking forward to your implementation. Consider creating a "library" like mine which might be included in other py-apps or act as a template for other implementations.. but I'm sure you do ;-)

[postlund]

@funtax It will be part of pyatv as a separate module once I'm done, so it will be possible to import it from pyatv in case that particular functionality is needed (and using this library is not an option) 😄 But it would be too excessive to create a new package and publish that to pypi for just this algorithm alone.

[postlund]

@funtax Can you tell me what the purpose of this snippet in doPairSetupPin3 is? 😮

int lengthB;
int lengthA = lengthB = aesIV.length - 1;
for (; lengthB >= 0 && 256 == ++aesIV[lengthA]; lengthA = lengthB += -1) ;

Comparing the value of a byte with 256 will never succeed since max value of a byte is 255. So what I can see, this loop only increase the value of the last byte in aesIV by one?

[funtax]

Hey @postlund Sadly no, I'm sorry. I have reversed another (obfuscated!) program and this was the line I didn't understand on the first go. Leaving it out didn't work so I just copied it 1:1 into Java-code.
I would have to check this in a quiet minute in the debugger if you don't "solve" it before :)

[postlund]

@funtax That's OK, just wanted to check if you knew the purpose. Using a decompiler can leave strange code like that, so I'm not really that surprised.

I also wanted to let you know that I've made quite some progress. Currently, I'm only focusing on the algorithms and doing prototyping. So it's far, far, far from usable in any way (I barely communicate with the device yet), I merely verify that passwords and checksums are generated correctly. Almost everything crypto related is finished, it's the last EDDSA verification left. So I'm starting to feel confident that I can pull this off, which is nice.

[funtax]

@postlund Incredible, glad to hear that! Once you pass behind pair-setup-#3 you are good to go!

In case you like to "chat" about anything related to this via e-mail, type the java-package-Webaddress into your browser and follow it until you reach the corresponding app on Google-Play and use the support-address there. I just don't want to publish mine here ;-)

[postlund]

@funtax Yeah, it's really nice! I actually have working verification now (corresponding to authenticate()) and everything except for pair-setup3 seems to work as expected. I think I mixed up some identifiers and that's why it didn't work when I tried yesterday, so it probably works as intended. Algorithm-wise it should only be generation of the private key left. I have not found anything similar to how EDDSA in the i2p package for python, so I guess I will have to play around with what I have. But my final step now is to generate a new key with your library, extract the private part and the seed and verify that they work with my library. If that succeeds, I can finally start refactoring and implementing support in pyatv. It's a lot of work left when also including documentation and tests, but it will really nice when it's done 😄

Absolutely, if I need any one-on-one help then I'll certainly contact you. I don't think I will have time to work any further on this for a couple of days though, we'll have to see.

[funtax]

Could someone with the latest ATVv4 capture the TXT-records of the device (.raop.tcp.local)?
Eg. with the "Bonjour Browser"-app on Android?

I have no ATVv4 here and would like to use the version-part to determine if authorization is required.
Unofficial ATVs (EZCast, software-receivers etc.) would fail if they don't support the authentication.

[jeanregisser]

@funtax

[postlund]

@funtax Another alternative is (probably) to use dmap.loginrequired in the server-info interface. I assume that has the same value as the bonjour information and is available across all device.

[funtax]

Awesome, many thanks for your help, @jeanregisser & @postlund !

@postlund Is this the interface under "/server-info"? Because this produces an "access denied (403)" on protected devices (and is used by other software to "check" if the protection is active.
@ViktoriiaKh Seems to have the solution for this: owntone/owntone-server#377 (comment)

Retrieving this info via the TXT-record would be of course the most straight forward way.

[postlund]

@funtax Oh, that's some details I didn't know. The best solution in my case would be to not have to rely on bonjour data. Mainly because this project is used in Home Assistant, where configuration usually done manually by the user. So it would be best if the platform there handled everything automatically. But then again, the support is for Apple TV and nothing else so I can implement it in such a way that only the Apple TV is supported.

Other fun news is that I have successfully ported all parts to python now. Both pairing and authentication works as well as key generation. So it should be smooth sailing now on! 😄

[funtax]

@postlund Yearh I'm happy to read this :))
For my project I will now simply enable the pairing for all AppleTVs except v2+v3.
V3 is the one those emulators are using, so excluding them in the pairing should be enough for me.

Happy casting!

[mar-schmidt]

So what does this mean for the end user in terms of changes from current functionality? :)

[postlund]

@Ronelius Feature-wise nothing really changes, it brings back AirPlay streaming functionality to all devices running tvOS 10.2 or later. In reality we can probably interpolate this to ATV4 and later (my bet is that ATV5 is shown at the WWDC keynote tonight).

[postlund]

So, I have pushed the start of the support. It's available in PR #88. If someone has time, it would really nice to get some feedback if it works or not.

Usage is quite simple, just to auth:

$ atvremote --debug -a auth
Enter PIN on screen: 9756
You may now use these credentials:
9884D120B45A4C12:78324C1F78D117EC8AAAA2F5F8CD48D799098A229428FD1673BD521D168F080B

The generated credentials must then be passed as an argument, otherwise play_url will not work:

atvremote -a --debug --airplay_credentials=9884D120B45A4C12:78324C1F78D117EC8AAAA2F5F8CD48D799098A229428FD1673BD521D168F080B play_url=http:https://www.sample-videos.com/video/mp4/720/big_buck_bunny_720p_1mb.mp4

It's also possible to just verify if a given set of credentials are verified:

atvremote -a --debug --airplay_credentials 9884D120B45A4C12:78324C1F78D117EC8AAAA2F5F8CD48D799098A229428FD1673BD521D168F080B verify_authenticated

As I said, any feedback would be great! 😄

[mar-schmidt]

A bit rusty, do you know how to install that pr using npm?

[postlund]

npm is for nodejs so that's not gonna work 😉 I would recommend that you pull the repo and use a new venv for this, so it does not collide with your working library. There's a script that does most of this, so basically this:

git clone https://github.com/postlund/pyatv
cd pyatv
git checkout -b device_auth origin/device_auth
./setup_dev_env.sh
source bin/activate
atvremove -a --debug auth

Then you can use git pull to keep updated.

[mar-schmidt]

what do you know... hehe 👍

I don't really know if I'm doing something wrong. But I'm getting "Unknown command":

[(pyatv)pi@raspberrypi:~/pyatv $ atvremote -a --debug auth
DEBUG: Discovering devices for 3 seconds
DEBUG: Auto-discovered service Vardagsrum at 192.168.1.3 (hsgid: 00000000-1092-dc12-7d1d-158da0c84566)
DEBUG: Aborting since a device was found
DEBUG: Ignoring 192.168.1.3 since its already known with HSGID
INFO: Auto-discovered Vardagsrum at 192.168.1.3
DEBUG: GET URL: http:https://192.168.1.3:3689/login?hsgid=00000000-1092-dc12-7d1d-158da0c84566&hasFP=1
DEBUG: _login_request: mlog: [container, dmap.loginresponse]
  mstt: 200 [uint, dmap.status]
  mlid: 9 [uint, dmap.sessionid]

INFO: Logged in and got session id 9
DEBUG: GET URL: http:https://192.168.1.3:3689/ctrl-int/1/playstatusupdate?session-id=9&revision-number=0
DEBUG: _get_request: cmst: [container, dmcp.playstatus]
  mstt: 200 [uint, dmap.status]
  cmsr: 93 [uint, dmcp.serverrevision]
  cafs: 0 [uint, dacp.fullscreen]
  cafe: False [bool, dacp.fullscreenenabled]
  cave: False [bool, dacp.dacpvisualizerenabled]
  cavs: 0 [uint, dacp.visualizer]
  caps: 3 [uint, dacp.playstatus]
  cash: 0 [uint, dacp.shufflestate]
  carp: 0 [uint, dacp.repeatstate]
  caar: 6 [uint, dacp.albumrepeat]
  caas: 2 [uint, dacp.albumshuffle]
  caks: 1 [uint, unknown tag]
  casc: 1 [uint, unknown tag]
  cavc: True [bool, dacp.volumecontrollable]
  casu: 0 [uint, dacp.su]

ERROR: Unknown command: auth

It looks indeed as I have got the latest code pulled:

[(pyatv)pi@raspberrypi:~/pyatv $ cat pyatv/__main__.py |grep "cmd == 'auth'"
    elif cmd == 'auth':

[postlund]

Hmm, that's odd. Can you do "which atvremote" in the shell and verify that it picks the correct binary?

[mar-schmidt]

[(pyatv)pi@raspberrypi:~/pyatv $ which atvremote
/usr/local/bin/atvremote

[postlund]

Yeah, that's the system-wide installed version and not the one installed in the venv. Did the setup-script succeed when you ran it? Also, you can try doing source bin/activate again to ensure the venv is active. After running the script it is not automatically activated.

[postlund]

@Ronelius Yeah, it's generally best to use one venv per application. But great to hear that it works! 😄

It will be a while until I can finish this and also extend Home Assistant with support. So, my suggestion is that you do that authentication manually (like you did) so you get valid credentials. Then you use the shell command component (https://home-assistant.io/components/shell_command/) to run atvremote from command line. Since you have to activate the venv it's probably best to create a small shell script that does the setup for you, like:

#!/bin/bash
cd ~/pyatv
source bin/activate
atvremote --airplay_credentials=XXX play_url=$1

I hope that works!

[mar-schmidt]

Thanks, will do :)

[postlund]

Since 0.3.0 is released now, this can be considered fixed! 🎉

[ingsaurabh]

@funtax great work, I am porting it to android version :) , you mentioned app that you reverse engineered, can you name that app?

[funtax]

Hey @ingsaurabh, it's "AirAudio" + "AirSpot".
Feel free to contact me via its support-address as I have already done the Android-implementation.
I have just not published those modifications to github ;-)

[funtax]

Oh @ingsaurabh you asked for the app/software I reverse-engineered and not the software it's now used for. I don't want to disclose this information here.

[ingsaurabh]

@funtax not a problem :P , so your android implementation uses socket or url connection

[funtax]

@ingsaurabh It's a socket-connection as normal URL/HTTP-connections won't do the job.
To get it running on Android <5.0, you need to replace some of the AES-stuff with BouncyCastle-implementations.
If you are targeting Android 5+, just grab my code and you should be done ;)

[ingsaurabh]

@funtax Have you tested this approach when security is selected as password, as in my case ATV5,3 pairing don't works and when I try to play something it throws 403 error code can you confirm this?

[funtax]

You are right @ingsaurabh , in my quick test with my app, device-verification AND passwort-protection seems not to work out-of-the-box. Maybe the HTTP-authentication has to be done before the actual pairing is made.

[ingsaurabh]

@funtax I have checked HTTP based auth also before pairing but that too fails with same 403 error and no header field to extract auth token

[postlund]

I have not verified, but could the password protection be this one:
https://nto.github.io/AirPlay.html#passwordprotection

[funtax]

Yes @postlund that's the password-authentication and done via a simple Digest-authentication.
My apps support this since the beginning and I think if Password+device-authentication is activated, then we do need to first Digest-authentication and then the pair-verify.

My apps are now doing first pair-verify and then the Digest-authentication which won't work.

I'm not sure ig @ingsaurabh really tried the digest-authentication before?

[ingsaurabh]

@funtax @postlund I did tried digest based auth before pairing(for airplay) but as mentioned in my previous comment it returns 403(Service forbidden) so doesn't returns nounce to proceed further.

[philippe44]

Hi - First, thanks for the work you've done here. I'm implementing a version in Perl (nobody's perfect :-)). So far the step2 does not work. The Perl SRP package does :
M1 = H(H(N) xor H(g) || H(username) || s || A || B || K)
which is what @funtax does according to comments (and code, I think) - is there a reason why you overloaded the computeClientEvidence?

I also noticed that you overload getSessionsKeyHash as well and could see a difference. If S = (B - (k * ((g^x)%N) )) ^ (a + (u * x)) % N then
your K = H(S | 0000) | H(S | 0001)
in Perl SRP, K = H(S)
That does not seem to me to be what interleave mode does. Can you confirm that the really different function is in this K calculation?

[postlund]

I don't think the evidence calculation should make any difference compared to the one bundled in your library. It seems to match the specification and I did not overload it when using srptools (for python).

Regarding K, you should use K = H(S | 0000) | H(S | 0001) as you said. It's the correct routine and that's a change I had to make too (see https://github.com/postlund/pyatv/blob/master/pyatv/airplay/srp.py#L70, S = premaster_secret).

[funtax]

Hey @philippe44, please don't trust the description inside my code but only the content of the code.
The code is copied together and I debugged the code until it worked and pushed it on github.

Every method implemented in my repository is actually used and the SRP6 won't work otherwise.
The "interleave"-code is also copied from other places and was required to work.

Checkout @postlund 's solution, this should be a good start for your implementation :-)

[philippe44]

Thanks @postlund and @funtax. I still receive an error when I'm sending the proof, so I'm not sure what's missing. Just a silly question. The plist items (pk, salt, proof) ... shall they be encoded in a special way? By default the Perl Crypt::SRP package use everything in raw format (I've tried hex, b64 but did not help and it seems that all is raw anyway). Crypt::SRP by default handles everything in raw. I was wondering why you do hex convert of pk and salt before calculating the proof

[postlund]

Everything in the plist-responses shall be in binary (raw) format and not converted to hex. I would highly recommend to log the various variables for a successful pairing either with @funtax library or mine, just so you have reference values. Then you input the same parameters in your code (e.g private key, PIN code) and pin-point when they differ. That's how I did and just trying to "get it right" the first time would never have worked.

[philippe44]

Thanks @funtax and @postlund. I'm good now. Authentication & Verification works in Perl. I've been pretty unlucky with 2 packages from Perl (SRP and Plist) that each head a nasty bug which derailed me for a long while. I know need to do a C version as well, at least for the verification steps

I'm also documenting the different steps needed for exchange the and I'll make this available later. This is a pretty big list of crypto functions needed!

[philippe44]

Just a quick follow-up: it's all working now. I've created a small doc that describes the protocol, in a format similar to http:https://nto.github.io/AirPlay.html. You can find it here https://htmlpreview.github.io/?https://github.com/philippe44/RAOP-Player/blob/master/doc/auth_protocol.html
Thank you both

[LionisIAm]

@philippe44 , @postlund
Does anybody know how works second part of this protocol? I Mean an AppleTV part. What should I answer on AirPlay clients requests? Thanks if any help.

[philippe44]

Can't you get that from the doc I've made?

[LionisIAm]

@philippe44 yes, sure. Even I am trying to make my server without pw (so we are starting from 10.2. RTSP session authentication) IPad didnt accept my answer. As I can see, I should respond "with a body containing 96 bytes of data." as I did, but dont know what is " remaining bytes (usually 64) are some data <atv_data>" so random bytes are not accessible.

[LionisIAm]

And also Iam receiving messageInfoDefaultHttpRequest(chunked: false)
POST /pair-setup RTSP/1.0
Content-Length: 32
Content-Type: application/octet-stream
CSeq: 0
DACP-ID: C30201934BF1D1F2
Active-Remote: 514593877
User-Agent: AirPlay/280.33BigEndianHeapChannelBuffer(ridx=0, widx=32, cap=32)
This kind of request, which is not discribed at your document @philippe44

[philippe44]

understood - at this point, unfortunately I don't know. It might be that <atv_data> is a signature of something and iPad verifies it, but when writing a client, I did not have to verify it. Only thing I can recommend is to verify that all your other steps are correct using the test vector I've included. There is such an amount of crypto which include many options that any small mistep makes it wrong.

Another thing: I'm not sure how you can start at 10.2 as what is done here requires that all other steps are done before. They must be done only once, but they must because the client will send you a request that relies on the fact that its pairing was accepted before

[postlund]

@philippe44 @LionisIAm I had a look at the code I wrote for MRP, which also uses SRP but with a proecdure more or less looking like the one used by HomeKit. The <atv_data> parts seems to be a bit weird to me and my guess is that it's encrypted data with a signature of some sort (as you said, @philippe44). In the MRP code, the data corresponding to <atv_data> is decrypted with CHACHA20Poly1305 using the previously derived shared key. Basically like this:

 def verify1(self, credentials, session_pub_key, encrypted):
      """First verification step."""
      public = curve25519.Public(session_pub_key)
      self._shared = self._verify_private.get_shared_key(
          public, hashfunc=lambda x: x)  # No additional hashing used

      session_key = hkdf_expand('Pair-Verify-Encrypt-Salt',
                                'Pair-Verify-Encrypt-Info',
                                self._shared)

      chacha = chacha20.Chacha20Cipher(session_key, session_key)
      decrypted = chacha.decrypt(encrypted, nounce='PV-Msg02'.encode())

      decrypted_tlv = tlv8.read_tlv(decrypted)
      ...

The variable encrypted here corresponds to <atv_data>. Since AirPlay uses AES in CTR mode instead of CHACHA20Poly1305, I assume AES is used here instead. So maybe it is possible to decrypt <atv_data> using the shared secret to derive some sort of identifier that normal iOS devices verifies, but our implementations still don't (because we don't know how it works yet).

You can find the SRP code for MRP here (still WIP): https://github.com/postlund/pyatv/pull/114/files#diff-0db40656ea4dba91f392656886afa836R100

[MikkelSnitker]

Has anyone figured out what <atv_data> contains?