0 00:00:00,000 --> 00:00:30,000 Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/322 Thanks! 1 00:00:09,320 --> 00:00:10,699 Hello, everybody, I'm Nick. 2 00:00:10,700 --> 00:00:12,829 I'm the security engineering lead at 3 00:00:12,830 --> 00:00:14,899 CloudFlare, and I'm going to 4 00:00:14,900 --> 00:00:17,119 talk to you today about Heartbleed 5 00:00:17,120 --> 00:00:18,710 if you guys aren't sick of it already 6 00:00:19,730 --> 00:00:20,719 after this long year. 7 00:00:20,720 --> 00:00:22,789 So specifically, 8 00:00:22,790 --> 00:00:24,259 I'm going to talk about my personal 9 00:00:24,260 --> 00:00:26,719 experience or my experience at CloudFlare 10 00:00:26,720 --> 00:00:28,789 with what happened during 11 00:00:28,790 --> 00:00:31,399 during Heartbleed, when it came out 12 00:00:31,400 --> 00:00:34,519 and what happened after the fact. 13 00:00:34,520 --> 00:00:37,099 So has anybody read this 14 00:00:37,100 --> 00:00:38,149 article? 15 00:00:38,150 --> 00:00:40,519 This is this is a snippet from 16 00:00:40,520 --> 00:00:42,769 The Verge, and it 17 00:00:42,770 --> 00:00:44,929 is kind of semi fictional account 18 00:00:44,930 --> 00:00:47,159 of what happened during the disclosure 19 00:00:47,160 --> 00:00:48,170 of of Heartbleed. 20 00:00:49,670 --> 00:00:51,739 I can kind of paraphrase a little bit 21 00:00:51,740 --> 00:00:53,839 how that conversation went, but 22 00:00:53,840 --> 00:00:55,429 it kind of kind of went a little bit like 23 00:00:55,430 --> 00:00:56,719 this, like. 24 00:00:56,720 --> 00:00:58,799 So do you use details? 25 00:01:00,020 --> 00:01:01,549 No, not that I know of. 26 00:01:01,550 --> 00:01:03,019 Does anybody use details? 27 00:01:03,020 --> 00:01:05,268 And that's for websites. 28 00:01:05,269 --> 00:01:06,269 Nobody use data. 29 00:01:06,270 --> 00:01:08,089 Grumped details. 30 00:01:08,090 --> 00:01:09,249 How about topics? 31 00:01:10,790 --> 00:01:12,889 Well, while my answer was, 32 00:01:12,890 --> 00:01:14,629 well, what's Attila's heartbeat? 33 00:01:14,630 --> 00:01:16,879 I mean, at this point, barely anybody 34 00:01:16,880 --> 00:01:18,919 knew what this was. It was very obscure 35 00:01:18,920 --> 00:01:19,849 feature. 36 00:01:19,850 --> 00:01:21,979 And the answer 37 00:01:21,980 --> 00:01:24,169 here was, was, oh, there's the stupid 38 00:01:24,170 --> 00:01:25,339 and and there's a bug. 39 00:01:25,340 --> 00:01:26,809 You should turn them off. 40 00:01:26,810 --> 00:01:29,419 Right. So recompile open SSL 41 00:01:29,420 --> 00:01:31,579 with openness to sell no heartbeat's. 42 00:01:31,580 --> 00:01:33,709 And that was that was kind of that 43 00:01:33,710 --> 00:01:35,929 was that. So heartbeat's were off 44 00:01:35,930 --> 00:01:37,609 for CloudFlare. We have a pretty simple 45 00:01:37,610 --> 00:01:39,529 architecture, so deployed it really 46 00:01:39,530 --> 00:01:41,659 quickly and the answer was, 47 00:01:41,660 --> 00:01:42,889 OK, looks good. 48 00:01:42,890 --> 00:01:45,079 Public disclosure should be around April 49 00:01:45,080 --> 00:01:46,189 9th. 50 00:01:46,190 --> 00:01:48,349 So I guess a lot of people had this 51 00:01:48,350 --> 00:01:50,449 question. But why tell CloudFlare? 52 00:01:50,450 --> 00:01:52,579 Well, let me just go real quick 53 00:01:52,580 --> 00:01:55,279 and describe what CloudFlare does. 54 00:01:55,280 --> 00:01:57,919 So it's a reverse proxy. 55 00:01:57,920 --> 00:02:00,199 So if you have a website, 56 00:02:00,200 --> 00:02:02,179 CloudFlare can sit in front of it and 57 00:02:02,180 --> 00:02:03,289 block malicious traffic. 58 00:02:03,290 --> 00:02:05,749 That's the sort of red X up there, 59 00:02:05,750 --> 00:02:07,969 as well as send cached content, 60 00:02:07,970 --> 00:02:10,459 static content. That's the bright orange. 61 00:02:10,460 --> 00:02:12,589 So it reduces anybody getting 62 00:02:12,590 --> 00:02:14,869 to your website that's malicious 63 00:02:14,870 --> 00:02:17,389 and reduces the load on your website. 64 00:02:17,390 --> 00:02:19,729 And for this to work, this cloud 65 00:02:19,730 --> 00:02:21,319 has to be closer to the visitor than it 66 00:02:21,320 --> 00:02:22,789 does have to be to your website. 67 00:02:22,790 --> 00:02:25,069 So we have this global network. 68 00:02:25,070 --> 00:02:27,559 So are our nodes are closer to 69 00:02:27,560 --> 00:02:29,119 the visitors and that's that's how it 70 00:02:29,120 --> 00:02:31,339 works. And there's over a million 71 00:02:31,340 --> 00:02:33,409 sites on CloudFlare, including banks, 72 00:02:33,410 --> 00:02:35,659 government websites, Bitcoin exchanges, 73 00:02:35,660 --> 00:02:36,979 almost every Bitcoin exchanges on 74 00:02:36,980 --> 00:02:39,079 CloudFlare, the ETFs 75 00:02:39,080 --> 00:02:41,149 website, Reddit. 76 00:02:41,150 --> 00:02:43,279 I can go on and on and on. 77 00:02:43,280 --> 00:02:44,449 So lots of sites. 78 00:02:44,450 --> 00:02:46,759 But what CloudFlare does is very simple. 79 00:02:46,760 --> 00:02:48,949 It's essentially three 80 00:02:48,950 --> 00:02:51,679 services or two DNS, 81 00:02:51,680 --> 00:02:54,559 HTP and ETPs, 82 00:02:54,560 --> 00:02:57,139 which is powered by open SSL 83 00:02:57,140 --> 00:02:58,519 and Engine X. 84 00:02:58,520 --> 00:03:00,589 So the architecture 85 00:03:00,590 --> 00:03:02,299 is very simple in that every machine that 86 00:03:02,300 --> 00:03:04,669 we have can serve every site. 87 00:03:04,670 --> 00:03:07,009 So thinking back to 88 00:03:07,010 --> 00:03:09,799 Heartbleed, this is actually 89 00:03:09,800 --> 00:03:11,539 you can see why this would be a really, 90 00:03:11,540 --> 00:03:13,939 really bad situation for Heartbleed 91 00:03:13,940 --> 00:03:16,369 to hit anyways. 92 00:03:16,370 --> 00:03:18,499 So it happened early 93 00:03:18,500 --> 00:03:19,819 April 7th. 94 00:03:19,820 --> 00:03:21,949 Ten, twenty seven open SSL publish 95 00:03:21,950 --> 00:03:24,169 their advisory and that hit 96 00:03:24,170 --> 00:03:26,059 Hacker News really quickly after that. 97 00:03:26,060 --> 00:03:28,129 Within half an hour it hit the front page 98 00:03:28,130 --> 00:03:30,349 and about an hour 99 00:03:30,350 --> 00:03:32,869 later we posted our standard 100 00:03:32,870 --> 00:03:35,029 CloudFlare customer sites are patched. 101 00:03:35,030 --> 00:03:36,889 You don't have to worry about it sort of 102 00:03:36,890 --> 00:03:38,989 post. And this is this 103 00:03:38,990 --> 00:03:41,269 was you know, it was 104 00:03:41,270 --> 00:03:43,009 it was a thing. It was it was a bug and 105 00:03:43,010 --> 00:03:44,569 it was starting to gain some steam. 106 00:03:44,570 --> 00:03:47,719 And then about an hour after that, 107 00:03:47,720 --> 00:03:49,219 there was a tweet from code name Con. 108 00:03:49,220 --> 00:03:51,559 And I think everyone can 109 00:03:51,560 --> 00:03:52,789 know what this is. 110 00:03:52,790 --> 00:03:54,259 This is that's the next site. 111 00:03:54,260 --> 00:03:56,989 This Heartbleed itself, it was branded 112 00:03:56,990 --> 00:03:59,449 and it came out to mass media. 113 00:03:59,450 --> 00:04:01,669 So this became a really 114 00:04:01,670 --> 00:04:03,739 big deal. Heartbleed Dotcom had a 115 00:04:03,740 --> 00:04:06,229 logo at the mainstream press 116 00:04:06,230 --> 00:04:07,159 Heartbleed virus. 117 00:04:07,160 --> 00:04:08,329 I don't know if you guys remember that, 118 00:04:08,330 --> 00:04:09,859 but people were saying, oh, there's a 119 00:04:09,860 --> 00:04:11,419 Heartbleed virus out there. 120 00:04:11,420 --> 00:04:13,519 And I knew I knew 121 00:04:13,520 --> 00:04:15,709 it got really bad when my my 122 00:04:15,710 --> 00:04:17,569 mother called me and said, what? 123 00:04:17,570 --> 00:04:18,570 What's going on? 124 00:04:26,230 --> 00:04:29,109 So this was kind of a big deal and, 125 00:04:29,110 --> 00:04:31,899 well, we were finished patching so 126 00:04:31,900 --> 00:04:33,129 well, we had some time to kill. 127 00:04:33,130 --> 00:04:34,130 What are we going to do 128 00:04:35,320 --> 00:04:36,369 at this point? 129 00:04:36,370 --> 00:04:38,139 There is we decided on three things. 130 00:04:38,140 --> 00:04:40,539 And one was to help 131 00:04:40,540 --> 00:04:42,459 keep this scanner that Flippo had from 132 00:04:42,460 --> 00:04:43,839 falling over. 133 00:04:43,840 --> 00:04:45,939 The second was to turn our network into a 134 00:04:45,940 --> 00:04:48,129 large honeypot to see what type 135 00:04:48,130 --> 00:04:49,479 of attacks or what type of scans are 136 00:04:49,480 --> 00:04:51,879 happening. And then the third was to 137 00:04:51,880 --> 00:04:53,859 decide or to figure out what we're going 138 00:04:53,860 --> 00:04:55,119 to do about our certificate's. 139 00:04:55,120 --> 00:04:57,189 We have a quite a few websites that 140 00:04:57,190 --> 00:04:59,919 use our service and many of them use SSL 141 00:04:59,920 --> 00:05:01,179 and we had about one hundred thousand 142 00:05:01,180 --> 00:05:03,399 certificates and revoking them was not 143 00:05:03,400 --> 00:05:05,499 really at this point. 144 00:05:05,500 --> 00:05:06,939 The day after disclosure, it wasn't 145 00:05:06,940 --> 00:05:08,799 absolutely clear that this was something 146 00:05:08,800 --> 00:05:09,919 that you had to do. 147 00:05:09,920 --> 00:05:11,979 So first, let's talk a little bit 148 00:05:11,980 --> 00:05:13,269 about this Heartbleed scanner. 149 00:05:13,270 --> 00:05:15,189 So Filippo, who's now a CloudFlare 150 00:05:15,190 --> 00:05:17,709 engineer, wrote this server and go 151 00:05:17,710 --> 00:05:19,749 and you type in your hostname and it 152 00:05:19,750 --> 00:05:21,819 scans it for her for 153 00:05:21,820 --> 00:05:24,279 whether or not it answers Heartbleed 154 00:05:24,280 --> 00:05:26,409 heartbeat requests that are malformed. 155 00:05:26,410 --> 00:05:28,179 So these are small ones around one 156 00:05:28,180 --> 00:05:29,619 hundred bytes so that it doesn't leak 157 00:05:29,620 --> 00:05:32,109 anything beyond your standard 158 00:05:32,110 --> 00:05:33,399 frame. It shouldn't leak any any 159 00:05:33,400 --> 00:05:34,299 information. 160 00:05:34,300 --> 00:05:36,189 You put it on us and then put it behind 161 00:05:36,190 --> 00:05:38,589 CloudFlare and shout out to 162 00:05:38,590 --> 00:05:40,779 Kyle Eisen from our team for helping 163 00:05:40,780 --> 00:05:41,739 keep this up. 164 00:05:41,740 --> 00:05:43,509 But this this is kind of what from 165 00:05:43,510 --> 00:05:45,249 Philippos server, what it looked like 166 00:05:45,250 --> 00:05:47,349 there was up April 167 00:05:47,350 --> 00:05:48,969 8th, up to two thousand requests per 168 00:05:48,970 --> 00:05:51,519 minute. So this was a very 169 00:05:51,520 --> 00:05:53,769 highly used tool. 170 00:05:53,770 --> 00:05:55,659 And that's nothing because 171 00:05:56,680 --> 00:05:58,059 this is the next two weeks. 172 00:05:58,060 --> 00:06:00,279 Two thousand is the the bottom tick right 173 00:06:00,280 --> 00:06:03,279 there. So up up to ten, 174 00:06:03,280 --> 00:06:05,379 ten to twenty thousand a day, 175 00:06:05,380 --> 00:06:07,599 a ten to twenty thousand 176 00:06:07,600 --> 00:06:09,819 scans per minute for the next 177 00:06:09,820 --> 00:06:12,219 two weeks. And it we held it up to 178 00:06:12,220 --> 00:06:14,019 two hundred million tests in the first 179 00:06:14,020 --> 00:06:15,459 two weeks. 180 00:06:15,460 --> 00:06:17,049 So with the scanner up and up and 181 00:06:17,050 --> 00:06:18,050 running. Yeah. 182 00:06:23,340 --> 00:06:25,259 And thank you to Filipa, wherever he is, 183 00:06:25,260 --> 00:06:27,479 he's somewhere in the room stand up. 184 00:06:27,480 --> 00:06:28,740 This is thanks for the tool. 185 00:06:31,880 --> 00:06:33,350 I don't I don't see him, he's somewhere. 186 00:06:35,450 --> 00:06:37,159 But in any case, this is this is what 187 00:06:37,160 --> 00:06:39,349 what he found in terms 188 00:06:39,350 --> 00:06:41,539 of Domain's, it 189 00:06:41,540 --> 00:06:42,529 was really bad at first. 190 00:06:42,530 --> 00:06:43,849 This is the night. 191 00:06:43,850 --> 00:06:45,949 This is two days after Heartbleed was 192 00:06:45,950 --> 00:06:47,239 originally announced. 193 00:06:47,240 --> 00:06:49,489 Up to 30 percent of the sites 194 00:06:49,490 --> 00:06:51,049 that he scanned were vulnerable. 195 00:06:51,050 --> 00:06:53,149 And this luckily cut down. 196 00:06:53,150 --> 00:06:55,279 And a lot of people use this in their 197 00:06:55,280 --> 00:06:57,529 automated testing to validate 198 00:06:57,530 --> 00:06:59,449 sites. But, yeah, it got down to a low, 199 00:06:59,450 --> 00:07:01,339 low number pretty pretty quickly. 200 00:07:01,340 --> 00:07:03,699 So now that that was up and running back 201 00:07:03,700 --> 00:07:05,809 at CloudFlare, we decide, 202 00:07:05,810 --> 00:07:07,579 well, what can we do? 203 00:07:07,580 --> 00:07:09,679 Well, we can log every heartbeat 204 00:07:09,680 --> 00:07:11,539 that we see that comes in with with a bad 205 00:07:11,540 --> 00:07:13,639 size. And while we can 206 00:07:13,640 --> 00:07:15,799 put that data in the shelf until now, 207 00:07:15,800 --> 00:07:17,299 and that's that's kind of what we did. 208 00:07:17,300 --> 00:07:19,399 So here are logs from the 209 00:07:19,400 --> 00:07:21,529 9th and sixty 210 00:07:21,530 --> 00:07:23,869 nine percent of them had a message size 211 00:07:23,870 --> 00:07:26,269 of sixteen three, eighty four, 212 00:07:26,270 --> 00:07:28,639 which you might 213 00:07:28,640 --> 00:07:30,769 know as the largest power of two, 214 00:07:30,770 --> 00:07:32,779 you can fit into assigned sixteen bit 215 00:07:32,780 --> 00:07:34,099 integer. 216 00:07:34,100 --> 00:07:36,229 But you might also recognize it as 217 00:07:36,230 --> 00:07:38,299 the hard coded value in the 218 00:07:38,300 --> 00:07:40,399 SSL test python tool that came 219 00:07:40,400 --> 00:07:42,739 out the first day, 20 percent were 220 00:07:42,740 --> 00:07:44,329 one twenty one and that was actually from 221 00:07:44,330 --> 00:07:46,789 Flippo site and University of Michigan 222 00:07:46,790 --> 00:07:48,829 maybe where you guys scanning on April 223 00:07:48,830 --> 00:07:50,089 9th or 224 00:07:51,590 --> 00:07:52,909 in any case there are a lot of requests 225 00:07:52,910 --> 00:07:55,009 that used the 226 00:07:55,010 --> 00:07:57,109 zero length packet, which is another way 227 00:07:57,110 --> 00:07:58,819 to just check to see if your site is is 228 00:07:58,820 --> 00:07:59,569 is vulnerable. 229 00:07:59,570 --> 00:08:00,570 But 230 00:08:01,880 --> 00:08:03,949 about a week later it was around the 231 00:08:03,950 --> 00:08:04,249 same. 232 00:08:04,250 --> 00:08:06,319 So if there were people who 233 00:08:06,320 --> 00:08:09,259 are mass exploiting this against sites, 234 00:08:09,260 --> 00:08:11,359 they were probably using just the basic 235 00:08:11,360 --> 00:08:13,489 SSL test Python script 236 00:08:13,490 --> 00:08:15,379 and around 20 percent were still 237 00:08:15,380 --> 00:08:16,519 Philippos tests. 238 00:08:16,520 --> 00:08:18,769 So if you can, you can do the math 239 00:08:18,770 --> 00:08:20,179 here. 240 00:08:20,180 --> 00:08:22,279 It's it seems that a lot of people 241 00:08:22,280 --> 00:08:23,719 were just scanning with Lebas test. 242 00:08:23,720 --> 00:08:25,729 It wasn't a lot of mass exploitation. 243 00:08:25,730 --> 00:08:27,799 And flipping the 244 00:08:27,800 --> 00:08:29,899 numbers back, you see that one percent 245 00:08:29,900 --> 00:08:31,849 of flippo scans were actually against 246 00:08:31,850 --> 00:08:33,139 sites on CloudFlare. 247 00:08:33,140 --> 00:08:35,569 So this is what the map looks like 248 00:08:35,570 --> 00:08:37,399 for where the attacks were coming from. 249 00:08:37,400 --> 00:08:39,199 I know IP maps are not really that 250 00:08:39,200 --> 00:08:40,339 interesting. They don't tell you that 251 00:08:40,340 --> 00:08:42,829 much. But this is 252 00:08:42,830 --> 00:08:44,869 this is where it is. There's some some 253 00:08:44,870 --> 00:08:47,599 strange spikes like in Iceland, but 254 00:08:47,600 --> 00:08:48,649 don't read too much into it. 255 00:08:49,910 --> 00:08:52,129 Now, the question is, and this is 256 00:08:52,130 --> 00:08:53,569 this is what we were thinking when it 257 00:08:53,570 --> 00:08:55,759 first came out was why was it 258 00:08:55,760 --> 00:08:57,319 really so dangerous? 259 00:08:57,320 --> 00:08:59,899 Why was Heartbleed so bad? 260 00:08:59,900 --> 00:09:02,419 Well, it's a kind of a layer six 261 00:09:02,420 --> 00:09:04,639 request that doesn't necessarily 262 00:09:04,640 --> 00:09:06,769 get logged. People don't log 263 00:09:06,770 --> 00:09:08,989 parts of handshake's very often unless 264 00:09:08,990 --> 00:09:11,329 you have a specific ID's rule 265 00:09:11,330 --> 00:09:13,429 or something like that. And it's really 266 00:09:13,430 --> 00:09:15,739 bad in that. Sixty four K of server 267 00:09:15,740 --> 00:09:18,259 memory can be filtrated 268 00:09:18,260 --> 00:09:20,359 by one request, and this has 269 00:09:20,360 --> 00:09:22,549 longer login info, session cookies 270 00:09:22,550 --> 00:09:24,619 and perhaps Tila's 271 00:09:24,620 --> 00:09:25,909 private keys. 272 00:09:25,910 --> 00:09:27,019 We didn't know. 273 00:09:27,020 --> 00:09:29,209 And if you kind of look 274 00:09:29,210 --> 00:09:31,279 at the diagram here, this is that's the 275 00:09:31,280 --> 00:09:32,299 heap right there. 276 00:09:33,620 --> 00:09:35,959 Everything that's above the request that 277 00:09:35,960 --> 00:09:37,639 if you have a request comes in, it gets 278 00:09:37,640 --> 00:09:39,589 put on the heap and anything that was 279 00:09:39,590 --> 00:09:41,899 previously removed from the heap is 280 00:09:41,900 --> 00:09:42,799 still sitting there. 281 00:09:42,800 --> 00:09:45,199 So we know that there are passwords, 282 00:09:45,200 --> 00:09:46,819 cookies. People were finding this right 283 00:09:46,820 --> 00:09:49,189 away. And the question 284 00:09:49,190 --> 00:09:51,469 was, would the key be there? 285 00:09:51,470 --> 00:09:53,299 Is the key going to set up above one of 286 00:09:53,300 --> 00:09:54,300 these requests? 287 00:09:56,150 --> 00:09:57,319 So we looked at the code. 288 00:09:57,320 --> 00:09:57,949 Right. 289 00:09:57,950 --> 00:09:59,989 And what did the code say? 290 00:09:59,990 --> 00:10:02,089 Well, it said this can't happen, at 291 00:10:02,090 --> 00:10:03,349 least not in Engine X. 292 00:10:03,350 --> 00:10:05,269 The key gets loaded right away and 293 00:10:05,270 --> 00:10:06,379 therefore gets at the bottom of the 294 00:10:06,380 --> 00:10:08,479 stack. And any 295 00:10:08,480 --> 00:10:11,359 time that you do applications or 296 00:10:11,360 --> 00:10:12,919 for requests coming in, they're going to 297 00:10:12,920 --> 00:10:14,209 be higher up on the stack and they're not 298 00:10:14,210 --> 00:10:16,579 going be able to read the original key. 299 00:10:16,580 --> 00:10:17,779 Right. 300 00:10:17,780 --> 00:10:19,879 And Engine X itself was single threaded. 301 00:10:19,880 --> 00:10:22,009 So if you have, it's not going to be 302 00:10:22,010 --> 00:10:24,079 able to catch something halfway in 303 00:10:24,080 --> 00:10:26,359 the middle of another thread doing 304 00:10:26,360 --> 00:10:27,409 an operation. 305 00:10:27,410 --> 00:10:29,599 So Open SSL 306 00:10:29,600 --> 00:10:31,969 has a big number library that they use 307 00:10:31,970 --> 00:10:33,409 and they clear the memory when they're 308 00:10:33,410 --> 00:10:35,569 done. So if you're doing a handshake for 309 00:10:35,570 --> 00:10:38,059 TLS, all the cryptographic 310 00:10:38,060 --> 00:10:40,129 material is going to be cleared by open 311 00:10:40,130 --> 00:10:41,130 SSL. 312 00:10:41,810 --> 00:10:43,369 At least that's what we thought. 313 00:10:43,370 --> 00:10:45,559 Right. And we 314 00:10:45,560 --> 00:10:47,089 we weren't sure. I mean, I just looked at 315 00:10:47,090 --> 00:10:48,659 some code and what do I know? 316 00:10:48,660 --> 00:10:50,989 So we launched the CloudFlare 317 00:10:50,990 --> 00:10:52,849 Heartbleed challenge. 318 00:10:52,850 --> 00:10:54,739 So it was a this was something that we 319 00:10:54,740 --> 00:10:56,270 did to crowdsource an answer. 320 00:10:57,380 --> 00:10:59,449 So we set up a standard Engine X, 321 00:10:59,450 --> 00:11:01,159 which was outside of CloudFlare. 322 00:11:01,160 --> 00:11:03,619 It was on a third party VPs, and 323 00:11:03,620 --> 00:11:05,449 it had the vulnerable version of open 324 00:11:05,450 --> 00:11:06,049 SSL. 325 00:11:06,050 --> 00:11:09,379 And we said to all of you guys, 326 00:11:09,380 --> 00:11:11,909 come and find it, see what you can do and 327 00:11:11,910 --> 00:11:13,969 to show proof, give 328 00:11:13,970 --> 00:11:15,679 us a message signed with that private 329 00:11:15,680 --> 00:11:16,680 key. 330 00:11:17,240 --> 00:11:18,739 What did we find? 331 00:11:18,740 --> 00:11:20,659 Well, for the first couple of hours, 332 00:11:20,660 --> 00:11:23,029 there was there was trolling and 333 00:11:23,030 --> 00:11:25,369 as you can see, somebody clapping 334 00:11:25,370 --> 00:11:26,299 because they did this. 335 00:11:26,300 --> 00:11:28,609 But basically 336 00:11:28,610 --> 00:11:30,409 anything that you post onto the page is 337 00:11:30,410 --> 00:11:32,029 going to be put into memory and the 338 00:11:32,030 --> 00:11:34,189 Ingenix. So people were 339 00:11:34,190 --> 00:11:36,049 posting private keys in there and they 340 00:11:36,050 --> 00:11:38,359 were posting what looked like 341 00:11:38,360 --> 00:11:40,099 you can see my name there, Nick, what 342 00:11:40,100 --> 00:11:41,449 looked like a password's file. 343 00:11:41,450 --> 00:11:43,099 So everyone was getting really confused 344 00:11:43,100 --> 00:11:44,839 getting all this. There's a private key 345 00:11:44,840 --> 00:11:47,029 in my Heartbleed question, but nobody was 346 00:11:47,030 --> 00:11:49,129 actually getting the key until we saw 347 00:11:49,130 --> 00:11:52,039 this tweet from a fettah 348 00:11:52,040 --> 00:11:53,040 and. 349 00:11:54,750 --> 00:11:56,909 We took a look, this is 350 00:11:56,910 --> 00:11:57,419 our office. 351 00:11:57,420 --> 00:11:59,579 It's me pointing out a television screen 352 00:11:59,580 --> 00:12:01,709 and yeah, he 353 00:12:01,710 --> 00:12:03,630 solved it. So congrats to or. 354 00:12:11,710 --> 00:12:14,709 And so he wasn't the only one. 355 00:12:14,710 --> 00:12:16,539 This was in the first twenty four hours 356 00:12:16,540 --> 00:12:18,489 or so, there were 12 people in the first 357 00:12:18,490 --> 00:12:20,319 48. About twenty five people had had 358 00:12:20,320 --> 00:12:22,689 solved it and got the real key and 359 00:12:22,690 --> 00:12:24,819 sent proof. 360 00:12:24,820 --> 00:12:27,049 So can you still 361 00:12:27,050 --> 00:12:28,059 private keys? 362 00:12:28,060 --> 00:12:29,379 Absolutely. Yes. 363 00:12:29,380 --> 00:12:31,809 And it was solved in under 10 hours 364 00:12:31,810 --> 00:12:33,459 at private keys can definitely be 365 00:12:33,460 --> 00:12:35,109 vulnerable. But another thing we did was 366 00:12:35,110 --> 00:12:37,239 we logged where in memory the 367 00:12:37,240 --> 00:12:39,489 Heartbleed requests were coming in and 368 00:12:39,490 --> 00:12:41,559 we compared that relative to where the 369 00:12:41,560 --> 00:12:43,809 private key was initially allocated 370 00:12:43,810 --> 00:12:45,879 and then they never overlapped. 371 00:12:45,880 --> 00:12:48,339 So how did this how did it actually 372 00:12:48,340 --> 00:12:49,340 get solved? 373 00:12:50,380 --> 00:12:52,569 Well, there 374 00:12:52,570 --> 00:12:54,009 was yeah, there was a second bug and open 375 00:12:54,010 --> 00:12:56,769 SSL who would have known 376 00:12:56,770 --> 00:12:57,770 looking through that code. 377 00:13:03,710 --> 00:13:06,019 If you dump the memory of of the request, 378 00:13:06,020 --> 00:13:08,359 all the places in red are where 379 00:13:08,360 --> 00:13:10,549 key private keys did exist at one 380 00:13:10,550 --> 00:13:12,349 point and it turns out some temporary 381 00:13:12,350 --> 00:13:13,999 variables were not wiped. 382 00:13:14,000 --> 00:13:15,439 This is the this is the code to clean up 383 00:13:15,440 --> 00:13:17,539 the mess. There's big name 384 00:13:17,540 --> 00:13:18,989 free versus big, dumb, clear, free. 385 00:13:18,990 --> 00:13:21,289 This is just yeah. 386 00:13:21,290 --> 00:13:22,879 It was just in certain cases in the 387 00:13:22,880 --> 00:13:24,229 Montgomery multiplication, they didn't 388 00:13:24,230 --> 00:13:26,809 clear up. They didn't clean the partial 389 00:13:26,810 --> 00:13:28,459 pieces. 390 00:13:28,460 --> 00:13:29,869 We can do a little bit of math just to 391 00:13:29,870 --> 00:13:32,029 show how people actually 392 00:13:32,030 --> 00:13:33,079 solved it with this. 393 00:13:33,080 --> 00:13:35,299 But RSA, you have a 394 00:13:35,300 --> 00:13:36,079 couple of different things. 395 00:13:36,080 --> 00:13:37,940 You have a public exponent e 396 00:13:39,260 --> 00:13:41,599 you have two primes multiplied 397 00:13:41,600 --> 00:13:43,279 together. They make the public key and 398 00:13:43,280 --> 00:13:45,349 the private exponent D and 399 00:13:45,350 --> 00:13:47,419 any one of the if you get any one 400 00:13:47,420 --> 00:13:49,639 of P, Q or or D, 401 00:13:49,640 --> 00:13:51,379 you get the whole private key. 402 00:13:51,380 --> 00:13:54,019 So what people did was they took 403 00:13:54,020 --> 00:13:55,369 every one hundred and twenty eight bit 404 00:13:55,370 --> 00:13:57,709 block that they saw in 405 00:13:57,710 --> 00:13:59,479 X filtrated sharply data and they just 406 00:13:59,480 --> 00:14:01,669 tried to divide it into the modulus. 407 00:14:01,670 --> 00:14:03,919 And if it divided, there 408 00:14:03,920 --> 00:14:05,389 you go, it's factored. 409 00:14:05,390 --> 00:14:06,919 And this is how nine out of the ten 410 00:14:06,920 --> 00:14:08,959 people solved it turns out that one of 411 00:14:08,960 --> 00:14:10,939 the prime factors is just sitting there 412 00:14:10,940 --> 00:14:11,869 on the heap. 413 00:14:11,870 --> 00:14:14,029 After tens of thousands of requests, 414 00:14:14,030 --> 00:14:15,889 you might look upon it. 415 00:14:15,890 --> 00:14:18,289 But one enterprising 416 00:14:18,290 --> 00:14:19,429 gentleman, Ruben Tsou, who's 417 00:14:20,450 --> 00:14:22,609 at University of Cambridge, he 418 00:14:22,610 --> 00:14:24,889 used a much cleverer method, which is 419 00:14:24,890 --> 00:14:26,929 coppersmiths attack, which is a Latisse 420 00:14:26,930 --> 00:14:29,269 reduction attack, where you only 421 00:14:29,270 --> 00:14:31,579 really need about 60 percent of 422 00:14:31,580 --> 00:14:33,769 one of the private keys to to to 423 00:14:33,770 --> 00:14:35,569 find it out. And this depends on the fact 424 00:14:35,570 --> 00:14:38,179 that the public exponent is small. 425 00:14:38,180 --> 00:14:40,429 So for performance 426 00:14:40,430 --> 00:14:42,589 reasons, the public exponent in RSA is is 427 00:14:42,590 --> 00:14:45,169 small so that any public operation 428 00:14:45,170 --> 00:14:46,879 is is really fast. 429 00:14:46,880 --> 00:14:48,979 But he solved it in only 50 430 00:14:48,980 --> 00:14:51,079 requests. So this was 431 00:14:51,080 --> 00:14:52,220 that was actually really interesting. 432 00:15:01,650 --> 00:15:04,079 So private keys are gone, right? 433 00:15:04,080 --> 00:15:05,159 What does that mean, 434 00:15:06,780 --> 00:15:08,100 revocation time? 435 00:15:10,710 --> 00:15:12,599 You know, the Internet was built for 436 00:15:12,600 --> 00:15:14,159 this, right? I mean, people who designed 437 00:15:14,160 --> 00:15:15,809 the peak, they said, yeah, you know, 438 00:15:15,810 --> 00:15:17,010 people are going to revoke 100 hundred 439 00:15:17,011 --> 00:15:18,599 thousand certificates in twenty four 440 00:15:18,600 --> 00:15:20,669 hours. This is just this 441 00:15:20,670 --> 00:15:22,019 this is how we designed the system. 442 00:15:22,020 --> 00:15:24,209 Right. And this is 443 00:15:24,210 --> 00:15:25,379 what sounds kind of reported. 444 00:15:25,380 --> 00:15:27,749 That's that's mostly us, actually. 445 00:15:27,750 --> 00:15:28,859 That's that's all the Internet. 446 00:15:28,860 --> 00:15:30,839 But that's mostly CloudFlare. 447 00:15:30,840 --> 00:15:32,459 And this is this is what it look like. 448 00:15:32,460 --> 00:15:34,379 The blue line is revoke certificates. 449 00:15:34,380 --> 00:15:36,329 And as you can see, that's April 7th. 450 00:15:36,330 --> 00:15:37,379 This is after Heartbleed. 451 00:15:37,380 --> 00:15:39,659 This is when everybody was revoking and 452 00:15:39,660 --> 00:15:42,419 then that green spike is is Clouser 453 00:15:42,420 --> 00:15:44,589 revoking. But once we 454 00:15:44,590 --> 00:15:46,649 revoked all these certificates, we 455 00:15:46,650 --> 00:15:48,989 found out that, well, it didn't 456 00:15:48,990 --> 00:15:51,089 really mean that browsers wouldn't accept 457 00:15:51,090 --> 00:15:52,739 these certificates anymore. 458 00:15:52,740 --> 00:15:55,019 And I can go into that 459 00:15:55,020 --> 00:15:56,819 really quickly. There are three methods 460 00:15:56,820 --> 00:15:58,919 to do this that were 461 00:15:58,920 --> 00:16:01,199 built for, you know, handling 462 00:16:01,200 --> 00:16:03,299 revocation and certificates and five or 463 00:16:03,300 --> 00:16:05,759 nine. And first is the certificate 464 00:16:05,760 --> 00:16:06,899 revocation list. 465 00:16:06,900 --> 00:16:08,999 And this is just a 466 00:16:09,000 --> 00:16:10,739 flat file with a list of certificates 467 00:16:10,740 --> 00:16:12,839 that are revoked and 468 00:16:12,840 --> 00:16:14,969 did as revoking one hundred thousand 469 00:16:14,970 --> 00:16:17,099 certificates, breaks the URLs 470 00:16:17,100 --> 00:16:18,100 Hecuba did. 471 00:16:19,350 --> 00:16:21,449 So the CRL from Global 472 00:16:21,450 --> 00:16:24,059 Sign grew from twenty two 473 00:16:24,060 --> 00:16:25,169 to five megs. 474 00:16:25,170 --> 00:16:26,170 Yeah. 475 00:16:32,370 --> 00:16:34,439 This basically does the CRL 476 00:16:34,440 --> 00:16:35,460 server, and 477 00:16:36,780 --> 00:16:39,029 lucky for this, lucky for global sign, 478 00:16:39,030 --> 00:16:40,889 CloudFlare was in front of their KRELLE 479 00:16:40,890 --> 00:16:42,779 server, but unlucky for us. 480 00:16:48,630 --> 00:16:50,489 Where we're used to that kind of traffic, 481 00:16:50,490 --> 00:16:52,349 but yeah, you can see here every three 482 00:16:52,350 --> 00:16:54,539 hours there are waves and this had to do 483 00:16:54,540 --> 00:16:56,669 with the cycles in which scrolls were 484 00:16:56,670 --> 00:16:58,829 updated and Microsoft Internet Explorer 485 00:16:58,830 --> 00:16:59,879 downloading them. 486 00:16:59,880 --> 00:17:01,829 And that was pretty rough. 487 00:17:01,830 --> 00:17:03,209 Anyways, Earl's broken. 488 00:17:03,210 --> 00:17:05,249 How about OSPI XP? 489 00:17:05,250 --> 00:17:07,348 Well, this is the online certificate 490 00:17:07,349 --> 00:17:08,318 status protocol. 491 00:17:08,319 --> 00:17:10,649 This is a kind of question answer. 492 00:17:10,650 --> 00:17:12,779 Is this certificate revoked, yes or 493 00:17:12,780 --> 00:17:15,269 no? And well, 494 00:17:15,270 --> 00:17:16,348 it's really broken. 495 00:17:16,349 --> 00:17:18,449 And Chrome has kind of known this 496 00:17:18,450 --> 00:17:20,219 for a while and stopped checking. 497 00:17:20,220 --> 00:17:22,769 But if you have hard fails, it does 498 00:17:22,770 --> 00:17:24,779 disallow you from using certain networks, 499 00:17:24,780 --> 00:17:26,669 especially with captive portals. 500 00:17:26,670 --> 00:17:28,859 And if you have a soft fail, 501 00:17:28,860 --> 00:17:30,479 somebody on the network can just kind of 502 00:17:30,480 --> 00:17:33,179 drop the ISP response and voila, 503 00:17:33,180 --> 00:17:34,709 the site is no longer revoked. 504 00:17:34,710 --> 00:17:37,139 So it really doesn't work 505 00:17:37,140 --> 00:17:39,329 or scale to the 506 00:17:39,330 --> 00:17:41,879 degree that we'd want it to in plain 507 00:17:41,880 --> 00:17:43,139 being requested from the browser. 508 00:17:43,140 --> 00:17:45,299 In that sense, I would say 509 00:17:45,300 --> 00:17:47,249 URL sets. This is Google's proprietary 510 00:17:47,250 --> 00:17:49,349 method that they 511 00:17:49,350 --> 00:17:51,419 basically collect all the 512 00:17:51,420 --> 00:17:53,549 kraals that they can from different 513 00:17:53,550 --> 00:17:55,469 certificate authorities and put them 514 00:17:55,470 --> 00:17:57,029 together and install them in the browser. 515 00:17:57,030 --> 00:17:59,189 So you get updates with the browser 516 00:17:59,190 --> 00:18:00,659 of these sets of scrolls. 517 00:18:00,660 --> 00:18:03,209 I shouldn't say all the scrolls, but 518 00:18:03,210 --> 00:18:05,429 it's specific certificates 519 00:18:05,430 --> 00:18:07,589 and this is kind of what we found out. 520 00:18:07,590 --> 00:18:09,809 They only do EV search and special of 521 00:18:09,810 --> 00:18:10,829 sorts, and 522 00:18:12,390 --> 00:18:14,159 if the browser doesn't get updated, then 523 00:18:14,160 --> 00:18:15,659 you're not going to get an update on your 524 00:18:15,660 --> 00:18:18,299 URL set, which is it's 525 00:18:18,300 --> 00:18:19,469 kind of bad. 526 00:18:19,470 --> 00:18:20,849 So CloudFlare challenge dot com. 527 00:18:20,850 --> 00:18:22,829 Once it was solved, we revoked it and the 528 00:18:22,830 --> 00:18:24,929 way it was, it was 529 00:18:24,930 --> 00:18:26,939 not an easy search. But Chrome did market 530 00:18:26,940 --> 00:18:28,739 as revoked because, well, they added it 531 00:18:28,740 --> 00:18:30,119 manually into JSON file. 532 00:18:31,230 --> 00:18:33,539 So none of the hundred thousand 533 00:18:33,540 --> 00:18:36,239 CloudFlare certificates were 534 00:18:36,240 --> 00:18:39,539 being marked as revoked by 535 00:18:39,540 --> 00:18:40,859 Google Chrome. 536 00:18:40,860 --> 00:18:43,229 So we we basically 537 00:18:43,230 --> 00:18:43,949 made a hack. 538 00:18:43,950 --> 00:18:46,319 And if you can read this, but it's 539 00:18:46,320 --> 00:18:48,449 the most efficient for line of 540 00:18:48,450 --> 00:18:49,859 C++ revocation. 541 00:18:49,860 --> 00:18:51,119 This isn't chromium. 542 00:18:51,120 --> 00:18:52,859 It revokes all of our certificates. 543 00:18:52,860 --> 00:18:54,809 This is a hackish this is not scalable. 544 00:18:54,810 --> 00:18:55,739 You shouldn't do it this way. 545 00:18:55,740 --> 00:18:57,929 But it was it was how 546 00:18:57,930 --> 00:18:59,279 it had to get done because there were no 547 00:18:59,280 --> 00:19:01,259 valid ways of of doing revocation. 548 00:19:02,550 --> 00:19:04,619 So, yeah, revocation is pretty 549 00:19:04,620 --> 00:19:05,549 much broken. 550 00:19:05,550 --> 00:19:08,339 And what can we do? 551 00:19:08,340 --> 00:19:10,319 Well, there's Schauder certificate 552 00:19:10,320 --> 00:19:12,299 expiration periods that could help that 553 00:19:12,300 --> 00:19:13,649 would at least help with the cartels 554 00:19:13,650 --> 00:19:15,779 because you don't have to be holding on 555 00:19:15,780 --> 00:19:18,029 to old certificates for very long. 556 00:19:18,030 --> 00:19:19,919 You can sort of shrink the size of the 557 00:19:19,920 --> 00:19:21,539 sets pretty quickly. 558 00:19:21,540 --> 00:19:23,729 Hoxby must staple is an 559 00:19:23,730 --> 00:19:26,009 extension that requires you to 560 00:19:26,010 --> 00:19:28,469 send the server to send the Hoxby 561 00:19:28,470 --> 00:19:29,699 response in the handshake 562 00:19:30,720 --> 00:19:32,129 that can help to certificate 563 00:19:32,130 --> 00:19:33,569 transparency, something else that's 564 00:19:33,570 --> 00:19:35,489 thrown out there to solve this. 565 00:19:35,490 --> 00:19:38,069 But none of these are 566 00:19:38,070 --> 00:19:40,169 widespread and none of these have been 567 00:19:40,170 --> 00:19:42,419 implemented. So something has to work 568 00:19:42,420 --> 00:19:44,009 for revocation. 569 00:19:44,010 --> 00:19:46,139 So I guess in summary, there are three 570 00:19:46,140 --> 00:19:48,059 things that we did after after 571 00:19:48,060 --> 00:19:49,060 Heartbleed. 572 00:19:49,920 --> 00:19:51,989 We kept the scanner from falling over, 573 00:19:51,990 --> 00:19:53,909 turned our site into a honeypot. 574 00:19:53,910 --> 00:19:56,129 And while we definitively 575 00:19:56,130 --> 00:19:58,499 answered that, yes, you have to revoke 576 00:19:58,500 --> 00:20:00,269 your certificates and there's no excuse. 577 00:20:11,470 --> 00:20:13,269 So there's a lot of takeaways from 578 00:20:13,270 --> 00:20:14,270 Heartbleed, right? 579 00:20:16,560 --> 00:20:18,699 I don't want to be the one to sort 580 00:20:18,700 --> 00:20:20,529 of tell everybody how to take away what 581 00:20:20,530 --> 00:20:22,599 to learn from this. But open 582 00:20:22,600 --> 00:20:23,769 source disclosure is hard. 583 00:20:23,770 --> 00:20:26,049 And this this really was the first 584 00:20:26,050 --> 00:20:28,359 one of the year of 585 00:20:28,360 --> 00:20:30,609 what turned out to many be many open 586 00:20:30,610 --> 00:20:31,779 source disclosures. 587 00:20:31,780 --> 00:20:33,849 And we did learn a lot 588 00:20:33,850 --> 00:20:35,890 of lessons of how to do that correctly. 589 00:20:37,090 --> 00:20:39,099 Other things that pointed out, which seem 590 00:20:39,100 --> 00:20:40,899 obvious to people, but they weren't 591 00:20:40,900 --> 00:20:42,999 obvious in or weren't in open SSL, 592 00:20:43,000 --> 00:20:44,919 which is features should be disabled by 593 00:20:44,920 --> 00:20:47,409 default. Nobody who installed open SSL 594 00:20:47,410 --> 00:20:50,169 one oh one wanted 595 00:20:50,170 --> 00:20:53,139 heartbeat's necessarily so 596 00:20:53,140 --> 00:20:54,640 turn-off features by default. 597 00:21:03,440 --> 00:21:05,719 Another thing is, while expect 598 00:21:05,720 --> 00:21:08,059 the unexpected, that's sort of obvious 599 00:21:08,060 --> 00:21:09,469 in computer security, that we didn't 600 00:21:09,470 --> 00:21:11,209 really learn that from Heartbleed, but it 601 00:21:11,210 --> 00:21:13,279 was it was definitely a shock when it 602 00:21:13,280 --> 00:21:15,859 came to 603 00:21:15,860 --> 00:21:17,959 other things, these attacks, a 604 00:21:17,960 --> 00:21:19,759 lot of the attacks on real sites like 605 00:21:19,760 --> 00:21:22,309 CloudFlare. So we saw quite a few attacks 606 00:21:22,310 --> 00:21:24,409 as as I mentioned, but a lot of them were 607 00:21:24,410 --> 00:21:26,029 still we're just scans from people just 608 00:21:26,030 --> 00:21:28,249 trying to see if their sites were alive. 609 00:21:28,250 --> 00:21:30,739 So that's that's a reassuring sign. 610 00:21:30,740 --> 00:21:32,779 Crowdsourcing was effective for the 611 00:21:32,780 --> 00:21:34,309 Heartbleed challenge. I couldn't find the 612 00:21:34,310 --> 00:21:37,039 private keys. And luckily, there are 613 00:21:37,040 --> 00:21:38,949 very smart people out there who were able 614 00:21:38,950 --> 00:21:41,179 to find it. Is anybody here in the in 615 00:21:41,180 --> 00:21:43,309 the auditorium winner of the 616 00:21:43,310 --> 00:21:44,869 CloudFlare challenge? 617 00:21:46,070 --> 00:21:48,379 Is anybody here? I don't see any hands, 618 00:21:48,380 --> 00:21:49,910 but anyways. 619 00:21:50,980 --> 00:21:52,989 I don't have my glasses, so 620 00:21:52,990 --> 00:21:55,089 congratulations wherever you are, 621 00:21:55,090 --> 00:21:57,429 and the last thing is, is revocation 622 00:21:57,430 --> 00:21:58,599 needs a solution. 623 00:21:58,600 --> 00:22:01,119 And the last 624 00:22:01,120 --> 00:22:03,099 conclusion from this is, is really, 625 00:22:04,240 --> 00:22:06,579 you know, support open SSL. 626 00:22:06,580 --> 00:22:08,319 It's it's. 627 00:22:18,130 --> 00:22:19,660 I messed up my microphone there, but 628 00:22:22,030 --> 00:22:23,030 thanks. 629 00:22:25,200 --> 00:22:27,209 No, really, I mean, this is this is part 630 00:22:27,210 --> 00:22:29,279 of critical infrastructure for 631 00:22:29,280 --> 00:22:31,379 the world and for not only websites, 632 00:22:31,380 --> 00:22:33,669 but as Zach here was telling 633 00:22:33,670 --> 00:22:35,969 embedded devices, and 634 00:22:35,970 --> 00:22:38,099 these guys need support. So please 635 00:22:38,100 --> 00:22:40,229 support open and sell and I'm done. 636 00:22:40,230 --> 00:22:41,230 Thank you. 637 00:22:52,320 --> 00:22:54,059 OK, quick announcement before we start 638 00:22:54,060 --> 00:22:56,429 the Q&A, if you are going to leave 639 00:22:56,430 --> 00:22:58,589 the room, please get up now, go 640 00:22:58,590 --> 00:23:00,719 out quietly without talking so we can do 641 00:23:00,720 --> 00:23:02,979 the Q&A nice and quickly. 642 00:23:02,980 --> 00:23:05,369 Also, you will be only able to leave 643 00:23:05,370 --> 00:23:06,570 the room at this point. 644 00:23:08,720 --> 00:23:10,139 Yes. If you have any questions, please 645 00:23:10,140 --> 00:23:12,419 line up at the microphones. 646 00:23:12,420 --> 00:23:14,490 We'll start with microphone number one. 647 00:23:16,950 --> 00:23:18,209 Thanks. Not really a question. 648 00:23:18,210 --> 00:23:20,549 Thanks. Thanks. OK, we really, really 649 00:23:20,550 --> 00:23:22,649 followed the progress 650 00:23:22,650 --> 00:23:24,439 even before the talk with your paper 651 00:23:24,440 --> 00:23:25,379 republished. 652 00:23:25,380 --> 00:23:27,689 And we just want to contribute a 653 00:23:27,690 --> 00:23:29,999 piece of missing information. 654 00:23:30,000 --> 00:23:32,399 So the first 655 00:23:32,400 --> 00:23:34,739 attacks, as you call them, 24 hours 656 00:23:34,740 --> 00:23:36,869 after the disclosure that were 657 00:23:36,870 --> 00:23:39,089 us, we weren't really attacking. 658 00:23:39,090 --> 00:23:41,819 We started working on the scanner 659 00:23:41,820 --> 00:23:43,979 about 20 minutes after the public 660 00:23:43,980 --> 00:23:46,169 information, and it was a top 661 00:23:46,170 --> 00:23:47,099 scanner in the region. 662 00:23:47,100 --> 00:23:49,169 So it was a scanner and 663 00:23:49,170 --> 00:23:50,849 used multiple different methods. 664 00:23:50,850 --> 00:23:53,489 We just as we went, we 665 00:23:53,490 --> 00:23:55,649 came to the conclusion that there 666 00:23:55,650 --> 00:23:58,199 might be firewalls to be deployed 667 00:23:58,200 --> 00:23:59,249 in between which 668 00:24:00,900 --> 00:24:03,089 which may stop some 669 00:24:03,090 --> 00:24:05,189 of the packets. So we use many 670 00:24:05,190 --> 00:24:07,529 different packets in one in 671 00:24:07,530 --> 00:24:09,599 every request. So that's also a scanner 672 00:24:09,600 --> 00:24:10,600 there, 673 00:24:14,520 --> 00:24:15,269 is there? 674 00:24:15,270 --> 00:24:16,270 No. 675 00:24:21,230 --> 00:24:23,299 That quick interruption, if you're 676 00:24:23,300 --> 00:24:25,639 leaving on the ground 677 00:24:25,640 --> 00:24:28,309 floor, please only use the front left 678 00:24:28,310 --> 00:24:30,919 and the front door. 679 00:24:30,920 --> 00:24:33,229 If you're up there, you can use any 680 00:24:33,230 --> 00:24:34,969 door you want. But if you're down here, 681 00:24:34,970 --> 00:24:37,069 please only use this door and 682 00:24:37,070 --> 00:24:38,359 this door to leave the room. 683 00:24:38,360 --> 00:24:39,360 Thank you. 684 00:24:41,690 --> 00:24:43,789 Does your bike work out well for 685 00:24:43,790 --> 00:24:44,790 you? 686 00:24:48,620 --> 00:24:49,639 Jim, thanks for the comment. 687 00:24:49,640 --> 00:24:51,859 I don't think I saw anything specific 688 00:24:51,860 --> 00:24:54,019 to you to to your your account, but 689 00:24:54,020 --> 00:24:55,249 yeah, it's great to know. 690 00:24:55,250 --> 00:24:56,599 So, I mean, it's one of those things it's 691 00:24:56,600 --> 00:24:58,319 really hard to tell from our perspective 692 00:24:58,320 --> 00:24:59,659 about the attack, whether someone's 693 00:24:59,660 --> 00:25:01,309 malicious or whether someone just 694 00:25:01,310 --> 00:25:03,649 scanning a lot of the 695 00:25:03,650 --> 00:25:05,059 the holes that we see don't have any 696 00:25:05,060 --> 00:25:07,309 information that identify and 697 00:25:07,310 --> 00:25:08,390 what the intent is. 698 00:25:11,240 --> 00:25:12,709 Microphone number two, please. 699 00:25:12,710 --> 00:25:13,939 Hello. 700 00:25:13,940 --> 00:25:15,589 I have a question to CloudFlare. 701 00:25:15,590 --> 00:25:17,779 Could you stop looking for 702 00:25:17,780 --> 00:25:18,780 users, please? 703 00:25:30,290 --> 00:25:32,449 You make the Internet more central every 704 00:25:32,450 --> 00:25:34,609 day, all the fancy home, but you switch 705 00:25:34,610 --> 00:25:36,799 there and come on to the network 706 00:25:36,800 --> 00:25:38,869 is so small you could handle it with 707 00:25:38,870 --> 00:25:40,429 that thing asleep, all of the bandwidth 708 00:25:40,430 --> 00:25:42,439 and you're blocking all the notes all the 709 00:25:42,440 --> 00:25:43,440 time. 710 00:25:45,170 --> 00:25:48,289 We're looking into solutions for 711 00:25:48,290 --> 00:25:50,689 the main problem is that 712 00:25:50,690 --> 00:25:52,759 there is a lot of spam 713 00:25:52,760 --> 00:25:54,139 on tour for Web sites. 714 00:25:54,140 --> 00:25:56,389 And right now filtering 715 00:25:56,390 --> 00:25:58,459 through the good and the bad is 716 00:25:58,460 --> 00:26:00,649 something that we're working on and we 717 00:26:00,650 --> 00:26:02,539 are focusing on that this year. 718 00:26:02,540 --> 00:26:04,309 So look for something coming up this year 719 00:26:04,310 --> 00:26:06,469 for Tor users like. 720 00:26:13,260 --> 00:26:14,909 Next up, we have a question from our 721 00:26:14,910 --> 00:26:17,099 signal Angel asked on IFC. 722 00:26:17,100 --> 00:26:18,100 Thank you. 723 00:26:18,480 --> 00:26:20,639 So given that maximum TLC record 724 00:26:20,640 --> 00:26:22,919 length is 16 kilobyte, how 725 00:26:22,920 --> 00:26:24,989 is it possible to even get back 64 726 00:26:24,990 --> 00:26:25,990 kilobytes? 727 00:26:29,740 --> 00:26:31,869 It's you can split you 728 00:26:31,870 --> 00:26:33,789 can split a request over multiple 729 00:26:33,790 --> 00:26:36,039 records, I believe. 730 00:26:36,040 --> 00:26:37,479 I think I'm pretty sure that's how it 731 00:26:37,480 --> 00:26:38,480 works. 732 00:26:40,720 --> 00:26:42,639 Microphone number one, please. 733 00:26:42,640 --> 00:26:44,339 What is your opinion on. 734 00:26:44,340 --> 00:26:45,340 I said 735 00:26:47,320 --> 00:26:48,320 go for it. 736 00:26:52,300 --> 00:26:53,589 I'm not sure for the most qualified 737 00:26:53,590 --> 00:26:55,089 person to answer that. 738 00:26:55,090 --> 00:26:56,889 To be honest. 739 00:26:56,890 --> 00:26:58,359 I think there's well intentioned. 740 00:26:58,360 --> 00:27:00,399 I think the community acknowledges that 741 00:27:00,400 --> 00:27:02,369 there are a lot of problems, us not 742 00:27:02,370 --> 00:27:05,649 necessarily in terms of maintainability. 743 00:27:05,650 --> 00:27:06,939 It's one solution. 744 00:27:06,940 --> 00:27:08,200 I'm not sure it's the correct one. 745 00:27:09,230 --> 00:27:11,139 Yeah, I think I think Libros itself works 746 00:27:11,140 --> 00:27:13,239 for its goal, 747 00:27:13,240 --> 00:27:14,890 which is OpenBSD. 748 00:27:16,030 --> 00:27:18,609 They have a portable version by now 749 00:27:18,610 --> 00:27:20,799 as well as they did with 750 00:27:20,800 --> 00:27:23,229 all of our other folks. 751 00:27:23,230 --> 00:27:25,539 And it's 752 00:27:25,540 --> 00:27:27,699 much cleaner and still 753 00:27:27,700 --> 00:27:29,889 open as the cell tries to support the 754 00:27:29,890 --> 00:27:32,049 way old systems 755 00:27:32,050 --> 00:27:33,999 like Windows, 16 feet and bullshit, you 756 00:27:34,000 --> 00:27:36,339 really don't need so down 757 00:27:36,340 --> 00:27:38,349 maintainability, you will remain for a 758 00:27:38,350 --> 00:27:39,350 while. 759 00:27:39,730 --> 00:27:41,919 Yeah, I agree that openness 760 00:27:41,920 --> 00:27:44,019 itself does support quite a lot of 761 00:27:44,020 --> 00:27:45,369 different platforms and some people need 762 00:27:45,370 --> 00:27:47,679 that. But the different forms 763 00:27:47,680 --> 00:27:49,419 of openness as well that have come 764 00:27:49,420 --> 00:27:52,089 recently, Libras and sell boring SSL 765 00:27:52,090 --> 00:27:54,249 are taking patches from 766 00:27:54,250 --> 00:27:56,349 each other. So I think the 767 00:27:56,350 --> 00:27:58,749 more people looking at this project, 768 00:27:58,750 --> 00:27:59,750 the better. 769 00:28:00,790 --> 00:28:02,169 We have time for one more question. 770 00:28:02,170 --> 00:28:04,219 Microphone number three, please. 771 00:28:04,220 --> 00:28:06,489 Hello. And both your talks, you showed 772 00:28:06,490 --> 00:28:08,559 numbers of vulnerable sites 773 00:28:08,560 --> 00:28:10,839 decreasing and increasing again 774 00:28:10,840 --> 00:28:11,529 afterwards. 775 00:28:11,530 --> 00:28:12,550 Can you explain that? 776 00:28:14,340 --> 00:28:16,989 So I think there are small bumps 777 00:28:16,990 --> 00:28:18,849 that were just people coming and going. 778 00:28:18,850 --> 00:28:21,819 I don't think we saw a lot of 779 00:28:21,820 --> 00:28:23,529 websites that became vulnerable. 780 00:28:23,530 --> 00:28:25,629 I think a lot of it is just pieces 781 00:28:25,630 --> 00:28:27,129 of measurement websites that came and go 782 00:28:27,130 --> 00:28:28,959 between different scans. 783 00:28:28,960 --> 00:28:31,209 I don't think there are any large jumps 784 00:28:31,210 --> 00:28:32,210 that we saw 785 00:28:33,280 --> 00:28:34,989 when it comes to the data that I was 786 00:28:34,990 --> 00:28:35,319 showing. 787 00:28:35,320 --> 00:28:38,469 This is from Philippos scanner and 788 00:28:38,470 --> 00:28:40,029 not everybody was scanning the same 789 00:28:40,030 --> 00:28:41,529 domains every day. So that's just 790 00:28:41,530 --> 00:28:42,579 standard variance. 791 00:28:46,120 --> 00:28:47,169 Thank you very much. 792 00:28:47,170 --> 00:28:49,269 Psychotronic, please give our 793 00:28:49,270 --> 00:28:50,890 speakers a warm round of applause.