Open Source Working Group session
Wednesday, 25th October 2017
11 a.m.
ONDREJ FILIP: Okay. I think we can start. Good morning everybody. Congratulations, you chose the right Working Group, this is the most funny Working Group at this RIPE meeting. If you want to see something boring, you should go to the anti‑abuse, that's the other room. If you want to see something fun about Open Source you are in the right place, welcome.
My name is Ondrej Filip. I will be chairing this session. Normally there is two of us here, but unfortunately Martin couldn't be here, we miss you Martin, so, Martin is watching remotely, I am sure he will comment and he will be almost like with us.
Before I start, let me just repeat the usual thing, which is the microphone etiquette. If you want to speak tell us your name and possibly affiliation if that's appropriate, because, you know, this session is recorded and stenographed.
One thing which is very important from the administration point of view. As I told you, we are two chairs, me and Martin, and every year we call for a nomination for another Chairperson, this happened before, and there was nobody interested and we both declared that we would like to continue if the Working Group ‑‑ the Working Group is happy with that. And there were no objections. So, I hope you are all happy with our work. If not, please speak up and tell us what's the problem. But, otherwise, we assume that we should continue for another year.
One thing we have today, quite busy agenda, we have three presentations and two lightning talks so we will surely fill all those 90 minutes, but before we start I need a formal approval of minutes from the previous Working Group, minutes were posted on time, so there were no comments so far, so if nobody will speak upright now, I will take the minutes as approved. I don't see anybody coming. Okay, and one before last, item for administrative matters, review of action points, we don't have any. There are no action points unless somebody has another opinion, there is nothing to do.
Before I introduce to you the presentations, a few names that are responsible for running this Working Group. Alistair is in front, he will take the minutes, he is the scriber of the session. We have Gabriel who is monitor the chat, so Martin, Gabriel will be your voice if you want to talk to us. And also we have other people helping us to make this happen, Stuart, and we have Mary as the stenographer. Thank you very much Mary, I wish you nice work today. That's all from administrative.
I will introduce the three presentations, the first one about Knot Resolver from Petter, and ‑‑ there is Debian Packaging for intermediates done by Ondrej Sury; Anotami white box series, two lightning talks.
So, I will not further delay the presentations because as I said the agenda is pretty full, and I will invite Petter to deliver the presentation about Knot Resolver.
PETER SPACEK: Welcome to talk about Knot Resolver with the main focus why it's different. So, first of all, what's Knot Resolver? If you didn't hear about it yet, it's just as many other basically, we have support for all the usual features. Which is not interesting right, BIND has it an Unbound has it, that's not interesting. What were different architecture than BIND and Unbound. It has other fancy features turned on by default, which is also very different from BIND. And in the end, I will explain why it's more like a platform than just a normal resolver.
So, shared nothing architecture. It sounds cool but what does it mean in effect? We decided that the usual bottleneck in networking daemons is the thing which is splitting requests from clients to different parts of execution, well we have one part in the system which already can do that quite effectively, right, and that's kernel, so, we are using the SO RE use port, so we don't need to split the requests ourselves so we have no bottleneck in here. Then, of course, we are a Knot Resolver and caching is absolutely essential, right, so even though we share nothing, all the processes are independent if you want, you can set up shared cache between those processes using the LMDB library which provides shared caches for all the processes, but still if any of those processes die, it doesn't matter. Rest of the process will take all the requests, and the resolution process will go on. And you can restart or terminate the processes at will.
That was the version of the architecture. How it works in practice. It's as easy as this. You are just put the number of processes with dash F and number. And that's it. Of course if you have system MD, everything is much simpler so you need to execute two different commands and I did some text files and so on but it was just down again to just appending ‑F, number of processes, and that's it. That's how we scale, it's as easy as that, no bottleneck.
As I said if you want you can have shared cache kiss shared amongst all the processes. You specify size of the cache and path to some directory where we can store the data for cache and that's it. No other special setup. If you decide to play with it, don't be surprised if you specify for example 100 MB, you will always see the 100 MB file on the system if you use L S, but the file is sparse, so don't be surprised, it's always the size is specified but in effect on this it just take some possible smaller amount of storage.
That was for the shared nothing architecture. I think it's done in the easiest way possible. And now about the 21st century features. The difference is that let's say traditional resolvers like BIND and Unbound are kind of afraid that they will break something, so, they don't enable the new features, which is for example the query name minimisation, which good for privacy, and random identification, which helps to increase entropy in the packets so it kind of helps with the come Inns key style attacks. So they have those two features enabled by default, which is the difference from BIND and Unbound and not the resolvers.
One very nice feature is unrun time reconfiguration, and in this month, we will have the version which will have the aggressive view of the DNSSEC validated cache enabled by default, which has super nice features when it comes to performance and mitigating some sub‑net of attacks against DNS for example, random some domain attacks. That's what we have enabled by default.
But if you want even more fancy features, we have automatic DNSSEC boot strap from IANA website and with verification built in, DNS 64, which might be finally relevant. If you go to IPv6 only, we have quite a nice DNS firewall REST API to configure the firewall, interface for the resolver if you want, so I strongly encourage you to play with it and give us feedback.
So, for run time reconfiguration. Every process of Knot Resolver has its own control circuit. If you have just one daemon, one process running, one socket, and you can connect to the socket and if you do that, you will see the common line interface, which is like right inside the resolver, and you can see and touch all the guts inside. You can do the boring stuff look like for the statistics, but if you really want, you can configure and change basically anything in run time. Let's say that you run the resolver and you realise oh I forgot to configure forwarding for this small sub tree or something, and to specify where sub tree configuration you need a policy module. Okay, so reload the policy module, why not, it's just library which we can load again, then you specify the policy ‑‑ I will not go to the details, it's well described in the documentation ‑‑ but I think is that you can change basically anything in run time, just connect to the circuit and change whatever you need.
That was for run time configuration. And now to the last point, which is like the main super important difference between the Knot Resolver and all the other resolvers. Basically, Knot Resolver can be seen as a platform for building recursive DNS service with all features you can think of. The recursive process is you can imagine how you resolve the different names and so on. And the nice feature of Knot Resolver is that it is exposing the state machine for all the users and developers, so you can see and modify the state of the resolver as much as you want and need. So, we have interfaces for I think modules in Lua and C and G languages, but basically you can write it in anything. Whatever interface we see, can be used to write modules for Knot Resolver.
I was talking about the 21st century features enabled by default, so I will illustrate how the module looks from inside on one example which we ship a long with the Knot Resolver, and the example is, in fact, module called work around, which is there for some super broken DNS providers which can't cope for, for example, with query immunisation or zero X 20 random identification or any change lower case to upper case will blow up because they didn't code their authoritative DNS so shame list is in source code. If you know anybody on the shame list, please tell them to fix their software. But anyway...
Typically, the module in the resolver has two or three parts. Configuration which is not relevant for work‑arounds because we know who are the bad guys so we can hard code it. And then you have set of selectors which basically say we are interested in modifying the hardware only for this sub‑net of queries, and then you have code which actually does the modification. I will describe that in detail.
So, in this example, we are concerned with PTR queries in the subdirectory where we are trying to translate a to a name and we know because we tested that some of the authoritative service of Turkish telecom are so broken that they couldn't possibly answer queries which have lower case and upper case mixed or if you have minimisation. So how do we might module which works out this bug on the authoritative side first we need to cut down the sub‑net of queries we are concerned with to the PTR. So we just check the query type, is it PTR or not? Let me do one step back, this is a Lua language, if you are not familiar, I will describe it. And I apologise for how it looks like on slides because I had to delete a lot of white space to fit it into the slide. In reality its much nicer than this. I apologise. So, first we check whether the query type is PTR or not and then if it's PTR we check whether the query is either finished or already failed. We are not interested in these because there is nothing to modify, it is it is ready done. If you find a query which is not done or not failed, we are now consenting the name of the name server which we are going to query for the PTR in Knot Resolver's line it's like waiting for cut and checking the name of the name server, we would query. So, if we don't have the name yet, we just let the resolver do whatever it would do and wait for a little bit until the server finds the name of the name server to contact.
So, assuming we already have a PTR query and we know the name of the name server we are going to query for the PTR, we can check whether it's on our blacklist, which in this example is a Turkish telecom, and if we are right now processing the query, PTR query destinated, two words, Turkish Telecom Service. We disabled the upper case and query minimisation because we know it's just broken. So it's as simple as that, you can see that's not really hard to write the module when you know which sub‑net of queries you want to affect, then you can do whatever you want.
Another little bit more complicated example which is the DNS 64 module which we ship with the Knot Resolver. It needs some configuration because you have to specify the IP address used as a prefix when you are translating IPv4 address to IPv6, so, if you specify something and we just convert AP address so configuration is not the hard part usually.
Then, again, we need to find or detect the queries which we want to change in some way. In this case we are looking for queries with query type AAAA, which ended with no data with no answer, but without any error, right. So, if the client queries a name and type AAAA and there is no answer for it from the author side, this DNS 64 module kicks in and tries to derive the IPv6 address from IPv4 address. So, this selector basically finds the IPv6 query without any data from the other side and if it's the case, it marks this query for further processing. Why is that? Because in this case, the resolver needs to fetch the IPv4 address for data to translate it to IPv6. We said like we will return back to this request from the client, and then generate a new query for the same name Type A, which is IPv4, and let the resolver to do the resolution without like assistance of the plug in developer. And then when the IPv4 address is received by the resolver, it calls the module again. The module will check the flag we set previously. So, we know that we are like in the middle of resolution of IPv6 address, and then we just transform the IPv4 address to IPv6. That's just the easiest bit work, so I will not show you on the slide.
So again writing modules is really easy for Knot Resolver. And if you want really something fancy, every possible service now has a rest interface right and we have it as well and to make it really easy to develop your own rest interfaces. So, with all the infrastructure which is available in Knot Resolver, you basically specify name of the end point, type of the data you want to accept and the function which gets called whenever there is a request received on that http end point and then the infrastructure and Knot Resolver will make sure that function is called and then you just do whatever you want with the http request basically. So you can conditionalise on the method, path and so on.
So to conclude the talk about the Knot Resolver, besides other things, it's really a platform for building recursive DNS service, so you can get really really fancy if you want. We have some time for questions now. So I think it's now microphone open for you.
AUDIENCE SPEAKER: So, does this also ‑‑ Lua interface, does it also expose the DNSSEC resolution results? My name is, Jelte Jansen, I work for SIDN. I have some really dirty hacks and inbounds to do some scary stuff and DNS resolution fails and I hope now that I have seen this that this could be done much better with knot, but I will try and play around with it, but... so I would need to know things like why it's failed and on which level. That's ‑‑
PETER SPACEK: Talk to me and we can look for the service in the code, it would be too long to explain from the floor. Basically all the guts of inside the resolver are exposed so you can see whatever you want.
AUDIENCE SPEAKER: Lots of space to shoot ourselves in the foot then. Thank you.
ONDREJ FILIP: Are there any other questions. When you are talking don't forget that those implementations must stay independent.
PETER SPACEK: If we have some minute or two ‑‑
AUDIENCE SPEAKER: Saif Al‑Mashadi, from CMC. I am asking that your software is compatible with BIND regarding its own transfer for authoritative DNS or its dotcom compatible.
PETER SPACEK: This is a resolver so it doesn't do its own transfer, right, but if you are asking about the authoritative site then sure, we are following the RFC, so if the input transfer is RFC compliant, than sure we can take it.
AUDIENCE SPEAKER: I checked your website that the ccTLD are using this software?
PETER SPACEK: Yes the authoritative server. Let me explain that the authoritative server is a different project which is called knot DNS and the recursive server is called Knot Resolver. I know it's a little bit confusing but these are two different projects with different code bases. I will explain that in the next slide if we have time.
So before we dive into knot projects, let me publicly thank Ondrej Sury who started the project. As you may have heard already, it was decided after eleven years and eleven months with cz.nic to move to ISC so now he is our competitor, so let's see how it flies. And because now he is gone and our competitor, we need to somehow show that we can deliver, right, even though we know him. So the both projects are going on under cz.nic wings, that didn't change, the knot DNS, the authoritative one is now led by Daniel Salzman and the Knot Resolver project is led by myself. And we need to prove that we can do something without Andre, so this month we released knot DNS Version 2.6, the authoritative one, which supports automatic DNS SEC algorithm rollover, so it's no pain any where are and nice features like inline signing on slave. So we basically can replace Open DNSSEC if you don't like it.
On the Knot Resolver side, the release is upcoming in the next weeks, and the main feature you can look forward to is, I guess the use of DNSSEC validated cache, which is super important if you are receiving a lot of garbage traffic for the route zone or some other zone which is DNS signed. So I recommend you to check out new releases and thank you for your attention.
(Applause)
ONDREJ FILIP: Thank you very much. And the next presentation will be from Ondrej Sury from ISC.
ONDREJ SURY: Hi, I am from ISC and this is actually a presentation I already had in cheque language, so, I translate it had to English, hopefully with no cheque is left, so ‑‑ the situation, you write an Open Source software because that's the way you write software, in Open Source. And you need to get it to your end users. And one of the ways is to like give them a source, but the other one is to just prepare packages for them. How do you do that? This is slightly more complicated than just for beginners interaction to Debian Packaging because I have been doing that for a long long time since thousand, everything seems easy to me. So, you need to excuse me a bit if it's complicated and you might ask questions and clarifications you want.
So, the contents of the presentation will be like, we'll show the structure of the binary and source packages and basic tool chain and the recommended tool chain, we'll make it more easy for you. And the one thing I really recommend is keeping the sources in GIT. And at the end, I would really welcome you to become a Debian developer or get involved in Debian at all, because we are run on volunteers and we need more fresh blood.
So, this is just to show you what I'm doing in Debian. This is my Debian portfolio, and I think that I am the longest service DHP maintainer and I think I am still not crazy. I also took over the lib jpeg, and took over all these, that's run by a crazy guy that's pushing non standard things into the I AG library, we moved to this which is faster and more standard. As all distributions have done. I also found it DNS packaging group and we Sheparded all the DNS, or most of the DNS resolver in the Debian. Obviously, we are packaging knot DNS, Unbound is made in ‑‑ GIT DNS, LDNS, power DNS has joined our Debian packaging group as well, since I became the new guy at ISC, I ‑‑ well told the chit ‑‑ moved Dubai 9 in the DNS packaging group as well. So I think we have a complete portfolio of DNS software in Debian in our packaging group.
That's a really good point ‑‑ good like entry point to join Debian to help the DNS packaging if you are interested in DNS software. The other stuff is Berkley DB, horrible stuff from Oracle. And BIRD, Cirus stuff, Apache, joined the Apache packaging group. I also help with these. If you are reasoning something on Debian, I am guessing you are using something package, I was will go involved in D N O ME, but it doesn't really play well well with the packaging it has different ideas how things are done.
Let's get to the package structures. The dot dab is shown in our archive and it's context of free files Debian binary, which is just a textual presentation of the formal that's ‑‑ because it was like one unannounced 20 but that's for a really really long time and then there is a control.star.GSE, it contains all the metadata for the package, the package information, the scripts that just run and after the installation of the package and like files like the check sums and config files. Then there is data archive and this is the actual contents of the package that ends up in your system.
So, these are the other translated for you. These are the tools for working with Deb files and it is the DPKG.deb, or you can just use the AR or TAR files and you can unpack the binary file the other way.
So, the source packet structure, this is ‑‑ usually it's just free files, and one file is like the package name, underscore and Debian version dot DSC, and this is like the description of the other file so it contains the miss announced stuff like maintainers, uploaders, list of binary packages contained in the source package and check sums. Then there is upstream TAR wall and actually from some DPKG version there can you be forum stream TAR balances but it happens very rarely. Then there are changes against the upstream TAR ball. Now it's simple TAR, previously it was.dif, but the TAR is the standard. It contains the Debian directory and in all the package performance when it was diff. It could contain the upstream sources but that's done differently now.
So, one interesting thing is Debian Packaging version, this can contain EPOCH, than its upstream version and after last ‑‑ it's Debian version, because Deb version can't contain the dash. The ipoc is for when you made a mistake in a versioning and you need to, like, go lower with the next number or you just replacing one binary package with something else like happened with GIT, because well the GIT was originally the new interactive tools. I very much doubt anybody in the room know what new interrupt I have tools, where. So, we need to like jump down with the version. So, you bump up the EPOCH and you can have the lower number, which is actually higher than the other one.
Then the upstream version is compared like that it's split to non‑numerical and numerical parts and then the non‑numerical parts wins over the numerical and it's like sorted. And the interesting thing is that if you use ‑‑ and now I don't know how the character is called ‑‑ tilda. The tilda is less than empty string. So, if you specify, for example, in the examples there ‑‑ here, the 0 tilda is less than zero, which can be used for such like, if you have a beta version or RFC version, then you can configure with the tilda and it will be lesser than the final release. It's usually the main use of that.
So, the source package, the Debian director, is the main, like, the entry point for the Debian packaging and there is several files, I will just list the most important ones, and there is Debian change log, this is the change log all the stuff you do to the package. So, it shouldn't contain the upstream changes but just packaging changes. Then there is Debian control, control, this is a metadata that describes the source package and the binary packages. Then there is Debian rules, and it's really make file but especially ‑‑ it's run with make. And it specifies the rules to building the package. Then there is Debian copyright. It's important because it's all about copyright. And other stuff. And last important thing that if you need to watch upstream sources, then there is a directory called Debian patches and it contains the patches to upstream sources.
So like the bear bone tool chain, because it's a make file, can be like this, but really don't do that. But it needs just four targets. It's clean, build, install and binary, and clean build is just return the source so a pristine state. The build will compile the sources, install will install the sources to a temporary directory and binary will assemble the binary package.
So, you can really do it like this, but it's not the way you should be doing.
So, if you want to do it the right way, there is a tool called DH_make, and this will create the basic skeleton in the Debian directory. It's not magic. It will not do everything for you, but it will do the most stuff for you, but you need to modify like write a description for the package and maybe modify the build rules. And it can, like, create a skeleton for several types of the packages that's single, this is just like you have sources and you create one single binary package. Then it's multiple binary packages of course, then there is binary, which is sort of multiple binary packages but there is a convention that the library is named lib, and basically you create the name from this name of the package. Who knows what the so name is? That's .so. Number, the first number, so it's lib name of the library, .so.number. So you just put this together and create the name and this is like basically the so name is the version of the library you are using and if you break the ABI, the binary interface, you need to bump the so name. This is why it's in the package name because you need to have those libraries constant for transitions.
Then there are some extra tools if you are packaging Perl style or Ruby style, you can use extra tools that will again create the basic skeleton for you. For the Perl you just type in the CPAN name for the GE M packaging, for the Ruby it's similar. It makes it easy for you.
So, this is the recommended tool chain. It's called Debian helper, and it's a set of dh_ something commands for packaging, like, dh automatic configure, autobuild, autoinstall, autotest and it will automatically detect the build system, if it's something done like auto tools or C meg or whatever, starter for Java , so it has also an automatic detection of the build system and it will do the most things right for you, so, the basic packaging is really easy. It has a compatibility levels. Now we are at level 10, which added some stuff. So, the level 9 I did the multiarchitecture support so you can have, for packages that's comply with multiarc, you can install the 32‑bit packages on the 64 bit system, if the CPU can run it. It has the hardening, which I will tell you about in a minute. The V 10 added some other new features, it's best to look at the page and also the composite levels and it will show you what's there. And it's extensible framework, unfortunately it's in Perl so you can only read in new modules. Since level 7, it added like only dh command which will do most things for you, and it will run a sequence of the dh underscore commands for you.
So here is the standard Debian helper sequence. I highlighted only the one you should be aware of, the other stuff is usually you won't get stuck in. But it's good to know. When you run dh binary, minus, minus, no act, it will just write all the sequence how it's one when building the package.
So, if you want to install files into package, then there is dh install DIRs, they will create the directories in the target ‑‑ like the target space for the package install and then there is dh install that will, that usually when you run like make install, then the Debian helper will make install into the Debian/TMP. And the install command, like will distribute the files in the TMP directory to do individual binary packages according to rules that are described in package dot install file in the Debian directory. It's country name files but since compatible 9 there is an extension called dh XX so when you mark the file as executable, its install file, it can run the ‑‑ it can rename files and it can use the variables. Here is an example. So this is the directly that contains the multi‑arc and it will move it to somewhere else. It's best to try it anyway.
So next thing is the documentation. The comment is called dh install docs. Again it's described in package dot docs, and it also installed the Debian copyright file and read me into the like the user will share dot name of the package directory. I will skip the part about the SIM linking.
So, if you want to tweak the standard behaviour of the dh command, there are targets called, you prefix it override, so if you need to override the automatic testing, for example, here I just prefix it with dash to make it not fail if the test fails, which might be useful, sometimes for testing, like because the Debian infrastructure runs on all different CPU architectures, then on some it might fail so you can, for testing, just not make the test failing and then just look up the resolves what's failing on what architecture, because Debian also has the big architectures and it sometimes fails.
There are some Deb helper extensions, there are extensions for cyst MG, for Python, for Ruby, I don't know, to update ‑‑ well ‑‑ I have it here. So some ‑‑ if you have sources generated in older systems that might not support newer architectures like R M 64 so you need to update the config gas and config sub files to support the new architectures. So this is do it for you.
Then there is Debian hardening. This is ‑‑ there are some flags for compiler like use format security, stack protector, position independent execution, and stuff like that. So, these are not enabled by default because it might break some code, but it's advised to enable them all if you can.
If you really got into packaging, I would recommend packaging, but keeping the sources for packaging in GIT. This is the GIT build package and it can install the upstream packages and then you will just type like GIT build package, build package and it will just check out the sources and build it for you.
So, the last important thing is that Lintien is a tool to check all the various errors in the package, and this is the most important page. How to contribute.
So I would recommend like fix a bug in a package or package something during your existing packaging team and help them. Write the coming, translate packaging into your language. There is a lot of stuff you can help, without being a Debian developer or maintainer. The other thing you can do is to adapt some existing package or create a new, for example, there are some packages that are funded by their maintainers because they no longer have interested or time to package them so you can help with that. And if you were like really like it and you do the work, you can become the Debian developer that has voting rights. But that's really not needed.
Here is some literature you should read. Debian policy manual, Debian developers researches, new maintainers guide and there is the thing about the GIT, packaging GITs from DKG. Okay. If you have any questions. Please go ahead.
JOAO DAMAS: Thanks for the presentation. This is the first time I have realised there is someone in the world that actually understands the Debian packaging system and how the hell it works. So, I was in the fortunate position of having to ‑‑ Debian packages a year ago, and the first I thought about killing myself then I just wanted to kill someone else. So, the documentation is there, it's true, but it's really hard to understand and then when you use it, it doesn't quite behave the way you expect it to. Perhaps if you have gone to the work of producing slides like this, would you consider putting together like a tutorial, something like Debian packaging for dummies like me?
ONDREJ SURY: Yes, why not. You can basically use these slides but go like ‑‑ I can talk about it for a whole day.
JOAO DAMAS: Possibly all the elements are there in the slides but a little bit more of a ‑‑
ONDREJ SURY: I can use these slides and talk for a whole day if you want.
JOAO DAMAS: You can talk and get someone else to type it down. That would be great.
Second question, a little more specific. When I was at the ISC, the Debian BIND package maintainer was someone called la month Jones, I don't know if he still is or not, but...
ONDREJ SURY: Fortunately for the BIND package, he like haven't had time to maintain the package, so when I took over I started to make things for modern. So it's not finished yet, but it's on the right direction. I just haven't had time to just kill all the old stuff, but it will be done. And there are more, like, two other guys just joined when we realised I took over the package, so, I have more hands. So it will not be just la month. He is still packager, and would I like to thank him for all the he did for the BIND 9 package, but we just moved the BIND 9 into the Debian DNS packaging group and we'll make it like up to the current standards.
JOAO DAMAS: I have a specific question and it's fantastic that you have your hands in both sides now. Since you have decided, you Debian people have decided to inflict system D on all of us, could you make the package work with system CT L & Co, like have the proper script, because it doesn't right now, it doesn't do that part right now.
ONDREJ SURY: I fixed some systemy stuff already, but it's not in this table, but if you have more like, please fill it up and I will take a note of it.
AUDIENCE SPEAKER: This is Gabriel from RIPE NCC, I have a question on behalf of Martin Winter. I found that the generic Debian packages are not very useful because of all the different variations in distribution and their differences. Problem is slightly different in packages available system VS in it dot D, classical DNS Ubuntu or older VS Debian. For the FRR routing package, we use back ports but this makes the setup and testing very painful. Do you have a better suggestion for a package across multiple Debian distributions or what is your experience there?
ONDREJ SURY: Let me think, I have it in the hardening slide. If you include this file, it will define all ‑‑ like the user ‑‑ it will dip fine all kinds of variables that describe the distribution you are building for, and then you can ‑‑ because this is make file ‑‑ you can like make if like conditionals, what so compile in or not. Well if Martin needs like advice, just, Martin write to me and I will help you with that. Because for some stuff, there is some distributions I still do back porting for doesn't have sis MD, so I have like these F delves in the make file, it works quite well if you have one source.
AUDIENCE SPEAKER: I still have one question. It's also from Martin. How do you deal with the testing of the packages of code which are not directly involved, how does a Debian maintainer usually get, know how a PE tension tool to test the packages. The question is from the view of an OS developer trying to best help a package maintainer on complex package like FR routing?
ONDREJ SURY: There are two ways. You can run the usual make check, but the other way is auto B kg test and that's an automated framework that's configured in the Debian directory, but I would have to have more time for this. And you can define tests that run over the finished package. Again, I can ‑‑ well if Martin writes to me I can help with that. Anybody can like stop me in the halls or write me an e‑mail and I will do my best to help. My time is limited so don't be offended I don't respond the first time.
ONDREJ FILIP: I wanted to say we have presenters, we are ‑‑ just, we have sometime for questioning as you shouldn't be discouraged. Just go on.
AUDIENCE SPEAKER: Stefan Jakob from DENIC. I have just to wish from maybe an upcoming thingy. For me it's totally okay to do all the stuff, but now we are in a world we have discontinued use integration thingy, so, what I would need is to make sure that all future pearls, if it ‑‑ continuous code streams coming in over time with the rest of my infrastructure that you can make sure that you can build the code on a continuous base without SLAS manual interaction if possible. I don't know what your experience is there, but from my point of already stepped away from Debian to ARC Linux or RPM based ‑‑
ONDREJ SURY: There is one thing called Debian Jenkins glue, I don't know if you have seen it. It will set up your infrastructure for you to build Debian packages from the Jenkins, and you can also like automate this with all kinds of stuff that Jenkins has, like, night triggers and get triggers and stuff like that, so this can be done.
AUDIENCE SPEAKER: If you have some experience maybe in a future talk I would much appreciate. How do you make sure that you have this continuous integration thing.
AUDIENCE SPEAKER: I wasn't going to say anything, but since it got brought up. Can you remove system D, that would be nice. Tim Armstrong. I just prefer if system D didn't exist any more. It kind of sucks really bad.
ONDREJ SURY: Okay, so you don't have a question, you are just drawling, right.
AUDIENCE SPEAKER: I guess. ...
ONDREJ FILIP: Okay. So we have another presentation from Alex Saroyan, thank you, so you break the line of the check people presenting. Thank you so much for that.
ALEX SAROYAN: Hello. I am Alex Saroyan from XCloud networks. At XCloud networks we build software defined networking platform which works on top of white box switches and I am going to speak about anatomy of bare metal switches, also called white box switches.
First of all, I would like to cover some major problems which we think from the point of well known vendor approach, so with well known vendor approach, that's the Cisco, Juniper and the rest of the proprietary stuff, that's very hard or sometimes impossible to get some customisations, imaging running some custom script inside the Cisco switch, that's absolutely impossible in most of the cases, of course that's no selection of operating system when you buy a Cisco switch it's coming with a Cisco operating system. When we compare this with the servers, usually when you buy some server from HP or somewhere else, no one is forcing you to run Windows server or some other operating system. If they do that, you are not happy, because you would normally like to install operating system of your choice on the server. So what's wrong with the switches that same could be and should be in my opinion with the switches. There is vendor lock in because sometimes when you buy some solutions from the proprietary vendors they are forcing you to stay with a particular hardware vendor and especially management software is in many cases vendor specific and sometimes companies are ending up of running different managing software for if they have different vendors using the network. So that always finally driving up the total cost of ownership.
So, this is a white box or bare metal switch. It's line rate, 1.44 terabits of switching capacity, 720 nanosecond of latency, with two power supplies and 4.1 fans. There are 48 ports and 6 times 4G of parts.
The switch is running on ‑‑ this particular switch is running on Broadcom, and just I want to make sure that everyone knows this that most of these big switches are running with the same basic, so the hardware inside of all of those switches comparing with the bare metal one I have shown is pretty much very similar.
But this switch cost from two to five times less comparing with the big verdicts.
Another very important thing that this switch allows to set up a network operating system of your choice. So this switch is coming kind of without operating system and technically you are able to install ‑‑ you can install some available operating system for example Cumulus, Linux or IP fix Ortechically you can port another Linux distribution, the one which you like better.
And these are the are verdicts who are producing this kinds of switchers, so HP, Dell and some others, these old guys are producing switches with a similar architecture, with a similar types of ports and so on, so the switches are kind of standard, but you have this big freedom of choosing among different hardware verdicts.
So you are not locking to vendor any more.
Let's look how is it organised. So you have front panel ports, like the ports SFP, SFP plus ports whatever, 10G, 40G, 100G. Then you have ASIC. The ASIC are mostly available from Broadcom or Mellanox. Then you have the CPU /RAM /Flash layer, this is a small computer inside the switch. This is the part where the network operating system is running. You have console, some RG 45 port and USB, this is for management of that port. The control plane is running there. You have network operating system, which could be Cumulus, Linux which we work very well, again that can be some other Linux operating system, you can even port your own operating system, in some cases I will cover this later on. And in the user level, you can run different ‑‑ because it's Linux, you can run different ways to control the switch that can be Puppet Ansible or Chef, you can use our software which is another option. And for the routing perspective you can use free range routing which is former so‑called Quagga, it is a fork of Quagga. You can run BIRD or you can run basically anything else. It's just Linux.
So, let's have a look how open are they? There is open compute project, which is backed by Facebook, Goldman Sachs and some other big companies. They are putting big efforts to make this open networking thing to happen. And the idea is that the network verdicts who are making this open networking equipment, they are placing all the scheme axe, gerber files, the gerber files are required for PCB manufacturing files, the BOM files and there are different types of licences, permissive and reciprocal. Many of the designs are permissive so you can really find ‑‑ imagine you want to build a switch, you can find all the required documentation and technical files which are required for building the switch.
Comparing different ASICs. Broadcom is the more popular: So when you design a switch with the Broadcom and you want to create the operating system, you need to sign an NDA with the Broadcom to get the drivers, with ‑‑ there is GitHub and you compile and integrate into your operating system. There are a lot of different options with the tried enter tool you can do 48, 10g port. With Broadcom you can do 30 to 1004G ports and with the others you can do a lot of different options and they even have the switch which is half unit, so spending a single unit within your URX space you can place even two switches. They can do like 120 K of IP addresses in the forwarding ways, and Mellanox can do 176k of IPs in the forwarding plain. Those limits vary depending on the setup.
The spectrum 2 is ASIC which can do 6.4 terabits of ethernet switching and IP routing. They can do 16 times 400g or, so you see this there is a lot of options. By default, this ASIC can store 512k of Mac, IP or ECMP records, and what is very interesting, partly for us, is that with the small extension with adding some additional external memory, this ASIC is able to do 2 million of IP routes. So, I imagine that after sometime that one switch will be able to handle the whole, full table of IP routes and I imagine that the one unit switch is swapping in some setups, the big Juniper border routers or whatever.
Another so‑called what we called killer ASIC. It comes with PISA architecture, which stands for Protocol Independent Switch Architecture, the main distinguisher here is that the ASIC, so standard ASIC is coming with the standard feature set, packet forwarding, and the rest of the functions are defined in the hardware and in control plane we do some computations and create some micro code which basically defines how the ASIC is behaving. With the barefoot networks, ASIC, this is a little bit different, because features which are supported by the ASIC is possible to change, so, there is a programming language called P 4 which stands for programming protocol independent packet processors. This is like very simple language like Python and it will be relatively easy for engineers to configure how the ASICs behaviour, it will be possible to change.
So, imagine you want the switch to do some weird things like to do routing using some strange field from IP packet, this will be possible with the barefoot ASIC.
Just a couple of slides showing how does this look like. So, this configuration is taken from Cumulus Linux console. So this is configuration of bridges. So I think most of you are very familiar with this. It looks like normal Linux configuration and it is very easy for people who are familiar with the Linux to move to this platform.
And this is part of BGP configuration for some people it looks like Quagga, for some people it can look like Cisco, but whatever it looks like the same, and this is from free range routing. And again most of the people are pretty familiar how to configure routing, which is very ‑‑ making people to change to this platform very easy.
From an operations and maintenance perspective, as I said it's like Linux so you can leverage most of the standard approach which you were doing with the normal Linux servers like Chef, Ansible, whatever you like, you can write your own code and put this into the switch which is very useful. Another option software with some different company develop and offer like a commercial software, but again here the value is that with we have a lot of different options and we have plenty of options here.
Something important which we have to keep in mind. It looks like a server but it's not a server. So, the features are limited to capabilities of the ASIC. So for example, with most of the ASICs, you can't do knot, but you can do everything which ASIC is capable of doing. So always keep in mind that when we put some configuration into very ‑‑ into the Linux, which we like a lot, it's not just like the normal Linux, there is a driver which will be compiling everything from the kernel into the micro code and the micro code will be sent to the ASIC and the ASIC is the part which is responsible for the actual packet forwarding.
That's it. Thank you. Any questions?
(Applause)
ONDREJ FILIP: Thank you very much Alex, we are sometime for questions, so...
AUDIENCE SPEAKER: Thank you very much for the presentation. Filippe Duke from Netassist. To be consist I would like to say something in regard to your presentation. I would like you to jump back to the slides about architecture, where there is Linux and so on. I mean the slide of architecture.
I was talking about the actual switch chips. You said that they are mostly Broadcom and Mellanox, I should say that they are almost the only. Broadcom and Mellanox and this is the huge problem. The problem of monopolisation of this field. The reason I am talking about here is the software. Exactly about Broadcom. I am really disappointed about their policy about the software. Without the software this is called idea comes into the whole, the actual unsupported something after four or two years, as long as the hardware is bug free it works fine, but unless you have the bug in the hardware and you have no way to fix it. You just doomed by the idea. This is why I can see that it is likely to be dangerous approach to have such fields. The example of barefoot networks make me more happy at the way they they implement some idea of making some packet processing language, but I'm completely unsure, I am just ‑‑ I was thinking about the approach when I was doing some research stuff, but I lost in too many what ifs.
ALEX SAROYAN: Thank you for interesting question. From our experience I can tell you that the ASICs themselves are pretty stable, so they are placing like many layers of testing, so, expecting a bug inside the ASIC, inside the Broadcom or Mellanox chip is very unlikely. It is, you can compare like with having a bug in the Intel processor, it's the probability is similar. And yes, there are some small differences between Broadcom and Mellanox, especially in the software part. As I said for the company who wants to integrate the driver, so Broadcom's driver comes through the NDA, through the special contract between operating system vendor and the Broadcom and the Mellanox approach is more open. That's pretty cool of course.
With the barefoot network, yeah, great comment, that is great approach. And this is like logical transformation for which we are waiting a lot. But again, still I hope there will be no bugs in the ASIC level.
AUDIENCE SPEAKER: Question from Jabber. From Martin Winter. With Broadcom even if switches use the same chip and hardware, isn't it the case that Broadcom licences by the feature and not all features are offered to all users? Other features are in the chip but unknown to verdicts because of the strict NDAs and can't be used. What is your understanding there? How are barefoot and Mellanox compared to this? Are they more open?
ALEX SAROYAN: Understood. From ‑‑ I can tell from our experience, sometime ago we faced some issues with some ‑‑ with some kind of limits and because of mentioned NDAs, these limits were not disclose and it was very hard to do kind of determining the calculation of this limit. And the solution was actually to do millions of tests. At the end we figured out what are the limits, but, yeah, this kind of NDA are making some kind of problems, I agree, and the Mellanox is more open from this perspective.
AUDIENCE SPEAKER: I think there is a big potential ‑‑ Stefan Duke from DENIC. A couple of months ago we started to replace Juniper security appliances by Open Source stuff because of reasons and this enables us to be in this new modern world where you think about infrastructure as a code, where network is a series part of it. Now we are able with this defined networking stuff, we are able to interconnect the software world our application world even with the network and so on. So this works pretty well. But there is a lot of stuff going on on this ASIC layer. Yesterday I had a discussion about this I square thingy, we are not able currently to read optical power budget and so on. So it would be really cool if the community is doing more pressure to the lower levels so that at least they have nice APIs or some sort of good documentation. Otherwise this is the trend, compute power is getting more and more powerful and I think for most of the common network cases you don't need a closed network any more and the bad thing is you can't really put it into the modern kind of infrastructure deployment. So it's really difficult where you need expensive software, interfaces and so on. I think we should push this more.
ALEX SAROYAN: Yeah, it's ‑‑ as you mentioned, it's a bit difficult, and when you want to develop something, when you want to develop your own software which works with this white box switches, then of course that will require some effort from you. But most important is that with this approach, it is, it requires efforts but it is possible. With older approach that was absolutely there was no choice actually. And I am happy that, besides us, there are some more companies which are putting this effort and creating software which is available for the people who don't want to ‑‑ who want to benefit but don't want to put in the effort that is not reasonable for them, and from the other side,s this is still available for people who want to go with efforts and get more narrowed and more specific benefits.
AUDIENCE SPEAKER: I think this is a common fallacy. Of course we have a big invest the first side, but afterwards, you save a lot of time and you save a lot of debugging work if you can automate it.
ALEX SAROYAN: Yeah, yeah, absolutely.
ONDREJ FILIP: Thank you very much, Alex. One more question.
AUDIENCE SPEAKER: Solomen Farsey. Actually, I am seeing in Europe, I see that you are using Intel and other providers for the hardware. Why you didn't go for the approach of having Open Source hardware description language like HTL or V log and using RPKI and other sources because in this case you can develop even more power into the system.
ALEX SAROYAN: Understood. Well, actually, so these two ASICs are coming with kind of fixed functions like you mentioned. And this ‑‑ so, this was coming actually historically and if they change the ASIC with programmable chips, which can use mentioned languages to programme, that would drive the prices very much higher, because of big computational power required from the chip. But, from other side, this is happening in the scope of barefoot networks. In the case of barefoot networks you are not programming with vary log, so it is not ‑‑ you are programming with the special language which is called P 4 and it's like vary log is very complex, you require a very, very special knowledge to be able. This is not a job for a network engineer. But what barefoot networks are doing, they are bringing this ability to the network engineering level. So, this is happening, but a bit different way.
AUDIENCE SPEAKER: The chip itself is programmable in hardware or as a CPU, built in CPU?
ALEX SAROYAN: Well, these chips, you simply push the micro code there and the micro code basically programmes different memory parts of the chip to define how the packet forwarding should happen. Like, you programme TCAM and whatever.
In case of barefoot networks, you programme behaviour, how should it work, and separately you push the micro code.
ONDREJ FILIP: That was the last question. Thank you very much Alex.
(Applause)
And we have a few minutes for short updates. So first will be Ondrej Sury from ISC.
ONDREJ SURY: Hi. This is actually the first time you see me presenting about BIND. So, be nice to me.
The BIND 9.12, we have just released the beta 1 last week and the final version should be around the end of this year. So this is a quick update what will be there. So, there are several new features. We will support the modern Crypto, the new ED algorithms but they need the open SSL 1.1.1, which is in development. But as long as they will release the new open SSL, we will support the new ED 25519 algorithm for DNSSEC.
The APNIC, thank you, sponsored the aggressive use of NSEC to reduce the unnecessary root queries. This the also be in the upcoming release.
Akamai has sponsored code to support the serve stale, this is the thing where you keep the entry in the cache in case you can't reach the upstream servers incation of a DDoS attack for example, it should help with the mitigation of the DDoS attacks on the resolver side.
And Tony Finch contributed a code to the DNSSEC key as a separate tool. These are the few new features:
We also modernised the code. So we are also moving to 21st century, slowly, but we are moving, and I hope to speed it up. Some of the most complex functions have been refactored and if you are interested in that in more depth, Evan Hunt had a great presentation at the DNS org last month so I invite you to watch the recording and see the slides.
Even one of the functions we refactored, we created the new library from just one function, that's how complex it was. You must understand that the BIND code has been started in '99, so it's quite old, it's the longest like running code in the DNS software, in the Open Source.
There was some other stuff like we removed SHA1 default for RSA in the DNSSEC key chain, and there is the dig plus SIG chase has been deprecated, the L W ‑‑ DLV support has been deprecated, it has been chopped down and added to the DNS org last month, so it's not needed any more, and we also ended Windows XP support. It was finally time.
So, one of the really great stuff that, one of the engineers has done for 9.12 was that he improved the performance stream a lot for a BIND. So this is the root zone performance, and the red dots is 9.12 and this is the rezone and because it's delegation heavy, there has been like 6 times the performance jump. But it had helped even in the case of TLDs, also delegation heavy. So this is the 1 million delegations it also helped. And we are also ‑‑ it also helped with just the hosting scenario, but not that much, but just a little bit.
As you can see, we had this drop in 9:10 and it has been fixed already, so if you are running some older version, I invite you to upgrade to the latest stable version, which is 9.11. Or even 9.12 because it will be in December.
So, this is the really like the things we had done for BIND 12 performance and I invite you to try the beta and report any bugs to us.
And because I'm new at the ISC, and this is really like an invitation for you guys to like talk to me, what would you like to see in BIND ‑‑ and I would really like to hear your feedback, what you think is good about BIND and what's wrong about BIND and what can I do about it and I will be here and I will be talking to you. So...
Questions?
AUDIENCE SPEAKER: First thanks for remaining available. One of the things I would always like about the ISC is it we mains available to have this conversation (Joao Damas (in the (on the refract erring, BIND BIND the kind of complicated, it does memory allocation, it does a lot of stuff. You are breaking this stuff apart into little bits. What about testing that it does what it used to do?
ONDREJ SURY: Evan had more on that. There is a quite more complex testing suit but we are still working on improving that so that we are reasonably sure that nothing had been broken by the refactoring. And in fact, I intend to refactor even more and one of my internal goals is to remove code, not add a new code in the upcoming BIND releases, which I think is the right thing to do. As they say the code is perfect when there is nothing to remove, not when there is nothing to add.
ONDREJ FILIP: Thank you very much Ondrej.
I know it's the lunch is coming so I have exactly five minutes and I will be very brief. I don't want to delay you from this bright part of your life.
I have a short update about BIRD routing daemon because I realised that the last update was more than three years ago. Just very briefly what's happening in this area.
Currently we have a version which is called 1.6.4. Some recent features are here on the slide. So we added BGP multipath. BGP large communities or large BGP communities, which you know, are now being discussed in various Internet Exchange point fora, because that's something that will be very, very useful when you are using route server and you want to signal where you have things including blackholing or peering policy and stuff like that.
We added one function which was, I would like to say, it's quite strange, it was there for a long time but MD5 authentication in FreeBSD was done partially, you could type a password in configuration, but it didn't work, you had to configurated in some other place in the operating system as well so that was not very handy so this was fixed in the recent version. And we had some new authentication in BFD, R.I.P., and OSPF, we used SHA1 and SHA‑2. We added completely new proposal and this was great because it was a submission from the community. We just kind of cleaned the code, cleaned the code and you know, redesigned it a little bit but it was reallied based on a patch from the community, so we have Babel and we also added IPv6 equals course multipath. So this is 1.6.3. It was released almost a year ago, so it's absolutely well tested release, rock stable. Although, we know there are some non critical bugs but nothing that will probably, you know, destroy your routing, so you can expect 1.6.4, but it will be just minor updates with a few fixes. But, this is very, very stable release.
What we are working on is a new version. Actually, so it's going to be called 2.0.0, we released pre‑0 and now it's pre‑1. This was raised hopefully six months ago, so we are intensely working now mainly on testing, bug fixing and finding all the problems. The main reason for this version was that we wanted to redesign the architecture of BIRD and currently you can use IPv4 and IPv6 together. In previous version, you compile BIRD as IPv4 daemon or with special switch as IPv6 daemon and they were independent. Now, with the new version, you can, for example, wire IPv4 BGP send IPv6 routes and so on.
The final release is expected this year, it's basically 2017. But you know, it's still under development, configuration may change. Why I'm doing this presentation is to kind of, you know, attract you to try to get some attention and ask you if you can test its release, if you can help on the testing, because you know, without any proper feedback it's quite hard to develop.
There is a list of new things, I don't think it's really important for us to read them. But basically many changes were done in BGP RIR relating to MPLS, VPNs and multicast. We added some BGP configuration and stuff like this. Maybe one feature which is although also related to BGP but it's not specifically BGP protocol, that we added the full support of RPKI router protocol, you know in previous version we had airway tables, but there was no mechanisms how to feed them in BIRD, now there is a protocol that can feed those airways inside BIRD you can see carriers for this.
So, so that's it. What's going to happen after. Again, we will continue on BGP, because, you know, we are doing Open Source, we are very much dependent on contributions or supporters and those are mainly from exchange point RIRs, so we will definitely work more on BGP, on especially BGP in very heavy environment like exchange point when there is one route server facing more than 1,000 peers for example, so we will try to increase the responsiveness and decrease the convergence in time. One thing that we promised and are definitely working on, which is one of the reasons why we integrated all those IPv4 and IPv6 address families, it's to create ISIS, so we'll be working on that and hopefully will be in some future releases. And I just wanted to explain that 1.6.X will be here for a longer time, it will be a long term supported, so don't be afraid that they will not fix security bugs there. If you tell me I am using 1.4, I will tell you please upgrade, but with 1.6 we will definitely fix all bugs there for a longer time.
And that's all I wanted to say. Thank you. Are there any questions?
AUDIENCE SPEAKER: Jabber: There is a mention from Martin. As an alternative to BIRD affect routing just released 3.0 major version a week ago with around 2,000 newcomers since Version 2.0 from six months ago. It was like he wanted to mention this.
ONDREJ FILIP: That's a good comment and I'm happy that there is a variety of other Open Source implementations, great to hear that. And greetings, Martin.
AUDIENCE SPEAKER: Stefan from DENIC. I am following the Bird mailing list for years now, but I am still wondering how you guys ensure the software quality and software security, because in the past there were some bugs were ‑‑ I am wonderring how you do guys make sure this important software, it's a really important software by now, how do you make sure the software quality that everything is going good?
ONDREJ FILIP: The good thing that many of our friends are exchange points and they are really dependent on this software, because BIRD is now deployed in more than two‑thirds of exchange points in the world. So, usually they help us when the busiest environment, so they submit to us the hints, what we should test, how we should set up the testing environment, so we have quite now intensive system of testing of interoperability of quality and everything. It doesn't say that there are no bugs, there is always a bug in a software, but maybe that's one of the reasons why we are a little bit slowed down the development because we concentrated more about the quality. I think I promised this version like two years ago to be honest and we spent really those two years improving, testing lab and all the tools related to that.
AUDIENCE SPEAKER: I'm not sure about the Open Source world, how you can make, for example, unit tests or something like this. I don't know if it's existing, but we don't have to discuss it here, but maybe putt it on the list, but if you have something to say this is really sophisticated what we do, maybe you can show us some day.
ONDREJ FILIP: Okay. Especially in Version 2, that's one of the examples how we improved the testing of the daemon.
AUDIENCE SPEAKER: Filippe Duke. A brief question, of course. We use BIRD a lot. There's a short question I have. How about your plans for better integration of net NS in BIRD. Thank you very much. Net dot name spaces to be ‑‑
ONDREJ FILIP: To be honest, that's the first time I hear such a request, so I don't think we have plans for that. So, I apologise, but if that's something which is necessary for the community, I am very happy to discuss it and bring it in in a future version. I apologise about not having better news for you.
Anyway. That's all. So, I will use the fact that I am here at the microphone and I will conclude the Working Group. Thank you very much for coming. I hope to see you in Marseilles. And let me thank all the Jabber monitors, scriber, stenographers and technical staff that were helping us to make this happen. Thank you very much and see you in Marseilles.
(Lunch break)
LIVE CAPTIONING BY
MARY McKEON, RMR, CRR, CBC DUBLIN, IRELAND.
