Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: CertStream – See SSL certs as they're issued in real time (calidog.io)
142 points by zer01 on Nov 3, 2017 | hide | past | favorite | 60 comments



A bit OT, but there is a thing about CT that I keep asking me and I can't seem to find an answer: Say I use a letsencrypt certificate for my nextcloud installation on my home server. Now say that the german police is investigating on me. They call up the BSI and tell them "hey, can you please issue me a signed cert for nextcloud.y0ghur7.xxx so we can mitm this guy? And please, don't log it to the CT, so nobody will ever know"

I will probably never know, because BSI is a trusted CA in all browsers and other http clients so they don't complain (and I am not looking at what certificate the server sent me every time I open my nc page), and nobody else will ever know that that certificate was released. Am I right? So what does CT buy me?


At some point browsers will stop allowing certificates that are not logged through CT


> At some point browsers will stop allowing certificates that are not logged through CT

Makes sense. So to be sure nobody issued a cert for one of my properties I would have to check regularly on CT logs to be sure that only certs requested by me are issued. But in that case, if someone requests a cert for one of my properties, and that cert was not requested by me, what do I do?

Do I tell mozilla and google that "someone issued cert id 4d8effdd25 for my nextcloud installation (or my forum where some rebellious users meet up sometimes) to mitm me, but it was not me". Will they belive me? And it will be probably to late anyway, because propagation to a CT log can take up to one day, so they got data on all the traffic for a whole day.


> Do I tell mozilla and google that "someone issued cert id 4d8effdd25 for my nextcloud installation (or my forum where some rebellious users meet up sometimes) to mitm me, but it was not me".

Yes, this is exactly what you should do. There's a very active list by mozilla ("dev-security-policy") where CA missteps are discussed on a regular basis, that's a good place to bring up all issues with CAs (however most of them are much more minor than a mitm attack with a fake cert - the day to day business is more "this cert violates RFC something").

> Will they belive me?

Well, the malicious issuance of a certificate is high profile enough that they will at least investigate and the CA will have to show some evidence how the cert has been issued.

> And it will be probably to late anyway, because propagation to a CT log can take up to one day, so they got data on all the traffic for a whole day.

That is in principle true. CT does not directly prevent attacks. But the general idea is this: CT makes it very likely that attacks get detected. A malicious attack by a CA is almost certainly the end of their cert business. So while an attack is still possible, it becomes very expensive, you basically have to sacrifice a working business.


Yes, it would be too late for you, but it would also be too late for the CA in this story, since the purpose of these technologies is to create and preserve a "smoking gun" and now everybody can see they aren't trustworthy.

In most countries, law enforcement are disinclined to use tools that will only work once - because what if they need that tool tomorrow for something more important? So this provides you with a bit of herd immunity, there is probably someone doing something naughtier than you who would be a better target.

It is also legally easier to get away with demanding that somebody do something they already _can_ do than to demand they come up with a way to do something they can't already do. British courts for example asked Internet Service Providers "Do you have a way to block web sites, e.g. for having child pornography on them?" and all the big ISPs said "Oh yes, we have that" and then the courts said "Aha. OK, then you must use that to also block copyright infringements, Hollywood will tell you what to block". But for the tiny ISPs like mine that said "No, we just move bits - nazis, child porn, bomb making, if it's illegal then you should convict the people doing it, not our problem" the courts said "Then it would be outrageous for us to demand you do as Hollywood asks, carry on as you were".

Because CT logging is mandated by Google, most CAs are building systems that automatically log everything. So then "Issue this but don't log it" becomes a huge ask, the front line guy the secret police get to says "I don't have a way to do that, it always logs everything" and that increases the chance the spooks get forwarded to an executive who says "Woah, this suicides my whole company, you better get yourselves a warrant, and I am calling my lawyer right now".


> I would have to check regularly on CT logs

You would indeed, which is part of the reason I released this service + libraries, so some enterprising developer can build a nice alerting service with it for folks just like you!

> Do I tell the mozilla that "someone issued cert id 4d8effdd25 for my nextcloud installation

Not exactly, I believe you'd probably contact the certificate issuer who issued the original certificate to have them issue a revocation, but my sincere hope is that folks running CAs will eventually come up with some better method for flagging certificates as bad/malicious than "just email Symantec support", since I wouldn't wish that on anyone.


>so some enterprising developer can build a nice alerting service

I launched https://ctadvisor.lolware.net/ some time ago.


Oh very cool! I hadn't seen this. Good stuff! http://cdn.ebaumsworld.com/mediaFiles/picture/718392/8489089...


Thank you! And yes, I definitely don't have a designer :p


one could also use crt.sh and the RSS feed.


> Do I tell the mozilla that "someone issued cert id 4d8effdd25 for my nextcloud installation

Basically. You'd send a report to the CA telling them there was a misissuance, and if their answer isn't to your satisfaction, you can report it to Mozilla and the other browsers on the public mailing list, claiming a misissuance by the CA. The browsers would then force the CA to follow up.


The CA in question is the German government itself (which happens to be trusted in all browsers).

They're almost exclusively used for digital ID servives (and, according to some sources, MitM), and are in browsers anyway. CT won't help with that.


The BSI is not the/a German police as you know. I don't think any of their certificates is trusted by browsers. The Bundesdruckerei certificate is though but neither are they a police force.

Do you have any source for that certificate being used for MitM?


The BSI is part of the executive, just as the police or the intelligence agencies, and its certs have been abused for that before.

There are a few stories about this in the 2013 Snowden data actually, the SPIEGEL published stories about that back then, too.


I have never heard of any of their certs being abused before and I have followed the Snowden revelations closely. The only thing I know of are some vague "cooperations with the NSA" that never have been described more closely. I don't think they even have a root certificate trusted by browsers. A publicly owned company (the Bundesdruckerei) does, however.


I have to be honest, I don't care that much about the details on this.

I'm far more enraged by the decision of the Bundesdruckerei to not offer on-card signing features for the ePerso anymore.

Instead, you have to use their web service to sign, and it costs you even more money. And is obviously ridiculously insecure.


Then please don't go around claiming "the German police has a trusted root CA and has used it for MitM in the past" if you don't actually know that.


It's not part of CT, nor does it fully solve the issue, but you might also like Certificate Authority Authorization. CAA allows you to publish what CAs are acceptable for your domain via DNS. CAs shouldn't issues certificates against that. Of course that doesn't protect against a rogue, compromised or coerced CA, but it does protect against phony requests to the CA.


As you said, that only protects against CAs that follow the CA/B Forum Baseline Requirements that require they check CAA at issuance time.

If a government was coercing a CA, they'd just tell them to disable this check. If this can be proven it's grounds to start the distrust process. At the very least, they should fail their next WebTrust audit.


It also doesn’t protect against compromised DNS, does it?


We're building this into the Cloudflare dashboard: the ability to monitor CT logs for issuance of certificates containing your domain name.


> "So to be sure nobody issued a cert for one of my properties I would have to check regularly on CT logs to be sure that only certs requested by me are issued."

CertSpotter does this for you automatically: https://sslmate.com/certspotter/


That's what the CAA dns records are meant to prevent. It tells Certificate Authorities which of them are allowed to issue for your domain.

Couple that with most providers requiring you to prove your domain via DNS or organizational status and you narrow the attack window.

Also I'd assume that as the owner of a domain, you'd be able to revoke any certificate for your domain that you didn't create.


That is mostly true, unless it's malicious Certificate Authority which may, on behalf of a governments request, ignore the CAA record on purpose to generate a certificate.

This is where a TLSA record would help to prevent malicious certificates. At least, if the client (browser) validates TLSA records.


Maybe that's the piece of the puzzle that's missing here? Being able to revoke a cert automatically by proof of domain ownership?


This is (imo) the most correct response to this query. The idea behind the CT lists is that browsers will flip into a "if it's no publicly identifiable, it's not valid" mode.

The Coinbase Engineering blog actually just put out an article on this very topic today - https://engineering.coinbase.com/moving-to-expect-ct-d0d26a0....


How can they efficiently do that? Will they send a request to Google/Mozilla/Firefox and asks if cert ABC logged?

I see two issues with that:

1) the browser vendors know each site and subdomain I visit. This seems to be a privacy issue.

2) every new visits adds a lot of latency because they need to check the certificate every time I request a site (now it becomes: dns, ssl handshake, cert check, http transfer).

3) when the cert check server is down, what is supposed to happen? Fail every ssl request? This adds a new point of failure. Just allow it? An attacked could black-hole the dns or block the IP address.


> How can they efficiently do that?

The web server sends a Signed Certificate Timestamp in the TLS Handshake¹. The browser will check that.

Apache support is coming², and other web server vendors are probably working on it as well.

¹https://tools.ietf.org/html/rfc6962#page-13

²https://httpd.apache.org/docs/trunk/mod/mod_ssl_ct.html


Even better is embedding the SCTs in the x509 structure itself so that you don't have to rely on obtaining/caching and the sending in the handshake. (Yes, there's some cases where a policy change my require the addition of additional SCTs—or different ones altogether—but this should be the exception not the norm.)



Similar to OCSP stapling the server could also include CT publication attestations.


> At some point browsers will stop allowing certificates that are not logged through CT

Chrome is scheduled to do start doing this in April 2018


Not answering the concert regarding CT (I concur).

But the obvious answer for truly secure connections is your own CA root or a self-signed cert you manually install/enforce.

You don't want to pass through another CA if your service is private. You actually don't need to. In fact, you'd want to manually enforce a single root or distrust system certificates entirely when doing so to avoid the problem you describe.

Most decently-written software that supports SSL/TLS allows you do that.

The CA system is for services that you intend to make public.


> But the obvious answer for truly secure connections is your own CA root or a self-signed cert you manually install/enforce.

> Most decently-written software that supports SSL/TLS allows you do that.

Unfortunately maintaining your own CA and installing it on all your devices is no easy feat. On android it's not even possible without root, and most IoT devices are not "decently-written". So what you say is true on paper, but delegating your CA to an external service like letsencrypt is unfortunately the only doable way as things are now.


There is Certificate Transparency for that problem. It is on work though.


At least google chrome (and im sure plenty of other browsers) embed some certs (or at least their hashes) in the browser to protect against this sort of attack for some services. See https://news.ycombinator.com/item?id=9078506 for some discussion on this -- its a slightly different response to a slightly different threat model.


That's cert pinning. I think you are talking about something else.


if you want to mit the guy you would need to re-route ANY traffic from nextcloud.y0ghur7.xxx to the BSI datacenter, I'm not sure if that is even possible. (also this does not find the owner of the site, how they do that you can read more about the dozens of takedowns of black market sites inside the onion network (most of the time it is/was human error))


One thing I totally dislike with CT is that literally everybody can see all the subdomains that my certificate is valid for (esp. LetsEncrypt), but also for cases where your "normal" wildcard-cert does not work - e.g. .foo.de is covered, but because wildcards dont go beyond 1 level, .bar.foo.de is not covered, and so everyone can see that there is one (or more) subdomains at bar.foo.de.

Let's assume an attacker finds a RCE in JIRA, Confluence or Gitlab... now everything the attacker has to do to find a list of candidates is to run a simple grep -i gitlab|jira|confluence|whatever on the CT logs, while he'd have to go the brute-force route before CT.


> grep -i gitlab|jira|confluence|whatever

Don't name hosts after products!

But more generally, you should never expose Confluence etc to the greater internet. That's just asking for trouble. It will find you, greppable hostname or not.


You should not expect a service which has been exposed to the open internet to be hidden.

If your last defense is that nobody knows the domainname, then you've lost. Not knowing the domainname shouldn't be any defense at all.


> You should not expect a service which has been exposed to the open internet to be hidden.

Of course not, but CT dramatically lowers the bar for attackers. That's what I mean.


It doesn't really, only when you need to hide subdomains for security reasons. Which you shouldn't do.

CT raises the bar for attackers since they will be logged into the CT if they try to MitM.

Any other attack is not made easier than without CT.


Soooo, in most circumstances these products are set to be allowed to be crawled by robots for search engines. I think it’d be a hell of a lot more accurate and powerful to use Google with keywords (like powered by lines) than say have to get into possibly vague hostnames (like docs, bugs, issues, git etc).


What you describe is called "Security by Obscurity" and it's not a good security mechanism. It is only applicable only after other type of defenses has been put in place.


Hey folks, developer of CertStream here. You can read more about the motivations and implementation behind this project by visiting the announcement page (https://medium.com/cali-dog-security/introducing-certstream-...) on my company's blog. I'm also happy to field any questions anyone may have!


Interesting project. I was wondering if you think they are any privacy issues around Certificate Transparency, like grouping ownership of domains through the timings.


Hmm, that's an interesting thing I haven't given much thought to.

I think that it would be somewhat difficult to pull off a correlation attack/leakage as the CTLs tend to dump in batches vs every poll returning new results, but I think once you remove a lot of the noise (cloudflare SNI certs, testing domains, etc) it'd potentially show some interesting patterns.

https://github.com/CaliDog/certstream-python/blob/master/exa...

This demo would be a decent starting point to that analysis if you'd be interested in toying with it!


Nice work! Congrats

How are you dealing with googles 60min-ish flushes of their log?


Interesting timing, considering the talk [0] on this very topic just uploaded to YouTube yesterday morning from DefCon 25. Basically, this is offering his observation (CTL can be used to get a real-time list of new domain names, which can be exploited), as a service.

Seems like Hanno Bõck could at least use a shout out if it was related to his work.

Either way, the talk is worth a watch.

[0] https://www.youtube.com/watch?v=TMNeSnjZfCI&list=PL9fPq3eQfa...


Shout out isnt needed, i did the same thing as Hanno over a weekend early this year. Been kicking myself since defcon that i didn't submit a talk!

Anyone who reads the certificate transparency log rfc can quickly realize whats possible.

I've also been following calidog since his first medium post, ive got my own similar cert scanner/tracker.


Interesting! I haven't been able to attend Defcon in the past few years so I haven't seen that talk (or heard of his research), but it's something I've thought about for a while now - using CTLs as a means to jump into the early process of setting up webapps and whatnot.

Thanks for the video!


There seems to be a big list of domains streaming through in alphabetical order. First there were dozens all starting with J, now there are dozens starting with K. Looks like gemalto is going through the day's domains in alphabetical order and adding them one at a time?


Yeah, it's interesting to see how certs are issued from the larger lists, since they tend to come in a deluge. This goes doubly for the cloudflare SNI certificates for their edge nodes!


Very cool! I love the simple JSON-over-WebSockets API and would love to see more streaming APIs like this available in the wild.


This is quite neat. I'll probably look at the go-library to build something out of that.

In the meanwhile, crt.sh also offers a Atom feed of issued certificates. Definitely not realtime but also works.


Sweet landingpage!


Thanks! It's a few iterations in, and my designer (http://www.jweiller.com/) was definitely responsible for the best looking parts!


heads up the <title> isn't getting set, it just shows the URL.


Thanks for the heads up! It also isn't displaying the favicon properly it seems :-/.

https://media1.giphy.com/media/R54jhpzpARmVy/giphy.gif TL;DR - Curse you Webpack! ::shakes fist::




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: