Routing Working Group session
Wednesday, 25 October 2017
9 a.m.

JOAO DAMAS: Good morning. Welcome. This is the Routing Working Group session here at RIPE 75. I hope that's the one you intend to be in. There is not a lot of people today.

Let's go quickly through the agenda which actually ended up being pretty packed.

While this gets sorted out. Let's get started with the administrative part. We circulated the minutes for the past meeting sometime ago, I can't remember how long ago. Does anyone have anything that they would like to amend? Because if they don't, then we will declare them final and put them ‑‑ declare them final as such on the website.

I'd like to start by introducing Paulo, my co‑chair and myself, both co‑chairs of the Working Group. We will go about that in point C in the agenda. I did forget the RIPE NCC who is going to take minutes this time around. Maybe someone will identify themselves, but usually the RIPE NCC is kind enough to provide someone to take the minutes as well as Jabber scribing. I don't see any hands. Let's hope for the best.

STENOGRAPHER: It's okay, you have me.

JOAO DAMAS: Thank you to the stenography system, without them it won't work at times we didn't have them around.

The agenda, the draft agenda there, there's not much wig he will room left but in case anyone has anything they would like to add this would be a good time to stand up and say so. No, okay. Gets go.

So, first real item with what we are calling the workshop Routing Working Group future revolution. If I could get the charter slide up...

So that's the current Working Group charter. It's been adapted through time a little bit. When we started it was all about RIPE database and how we expressed routing in the RIPE database database which later introduced the aim Ecosystems to the mix and again we updated again a few years ago. Well, this should be a dynamic thing. The world changes all the time, so this is sort of a chance for us to interact with you and get some feedback ideas for what the Routing Working Group should be looking at. Some people have come to us to express ideas. So if you could come up to the microphone and express them to the whole group now, that would be great.

AUDIENCE SPEAKER: So, thanks for bringing this up. And from my perspective, a few of the topics of interest what this Working Group can focus might be an a close interaction and alignment with the standard organisations in particular with IETF. There is a rather wide spread opinion that IETF as an organisation is disconnected from operational reality, and to an extent that is true, and one of the reasons for this is that the communication between the operations community and the IETF community is not necessarily that optimal. The groups of people who attend both events are not that much intersecting. So this leads to a situation where something is coming out of IETF and it is not necessarily practically deployable. For what reason? For very simple one, that the requirements were not necessarily formulated and brought into the IETF of the actual problem which needs to be solved.

So, personally I would see that Routing Working Group here might be a place which aggregates the requirements coming from operations community and tries to feed that into the relevant Working Groups in the IETF.

One other work area might be related to the Internet Exchange operators. The work starting there on trying to define the community's use for signalling of various options of the route server operation, and in order for this to be practical, applicable, it should be broader than just a set of regional acts and operators. It needs a more global view for that. So, personally, I would see this as one of the work areas that this Routing Working Group might be looking at.

JOAO DAMAS: Okay. Thank you. So, would I like to hear what everyone else has to say. Personally, I think this is a good idea to have this sort of interaction. Rudiger.

RUDIGER VOLK: I haven't looked at the charter for a long time and I would guess, scrutinizing what's in the working charter and comparing it to what actually happens in the Working Group, one would find a fairly serious Delta, and part of the Delta may be fine and just should be recognised, and some other parties of the Delta might actually point to deficiencies of what we are doing here.

For IGNOS, well, okay, in the good old days when I was running a different Working Group, I think I made a standard point in my agenda short report of what had happened in the IETF relevant, I haven't seen that happen I think in any of the Working Groups as a regularly scheduled item, and that should be one part of the communication.

For the other direction, I feel like recent history of this Working Group, I'm not sure ‑‑ well, okay, the large communities work actually had serious input and this Working Group was a venue that helped to formulate the stuff. But I am ‑‑ I somewhat feel that not very much of operational requirements have actually been brought up in this Working Group over the past, the relevant past, and I'm not sure whether that's going to be fixed in big ways.

JOAO DAMAS: If we listen from people, then we'll try to fix it, right ‑‑ we meaning and extending we, because I am planning to step down.

AUDIENCE SPEAKER: Geoff Huston, APNIC. Certainly in recent times we have seen, as Rudiger pointed out, large communities, graceful shut down, there's been a couple of small incremental changes in BGP which have actually been solely on the basis of operator input rather than just the IETF invention. The things that are kind of operating on many folks minds is actually the problem of route leaks, which is subtly different from protocol correctness, you know, the leak is actually about routes moving in contravention to policy rather than in contravention of the normal operation of the protocol. And to be perfectly frank, and I think you are going to discuss it in the last agenda item, there is this whole issue of what is the security issue in BGP and in inter‑domain routing and what are our alternatives, because BGP SEC appears to be relatively heavy weight and operators are not exactly happy with deploying that on Macs and that leaves open the question where does the acceptable compromise point move? And that is largely a question to operators rather than to vendors and designers. And it's a discussion that if you are going to have it anywhere, you might as well have it here.

JOAO DAMAS: Okay. Thanks, agreed. Anyone else? Okay. Well, what I hear is basically a good feeling about enhancing this sort of communication back and forth. And specifically the part that goes from the operators to the vendors, to the protocol definition part, because the other one as you say, we have had it here, sometimes people come here report about what is happening, but the other side of the equation has been missing so enhancing that would be great, that's valuable feedback certainly.

I guess the chairs to be will be able to incorporate that input. So, moving onto the next agenda item.

This is normal delay, formally people step down and while the next one steps up, they change the slides. But since I am staying here, that's why you notice the delay. Normally you are distracted by people moving around.

So, this is the next agenda item. The Routing Working Group Chair selection. As I said last time, this time it's for real, I am stepping down, I have been spending less and less times doing routing things and for time doing DNS things. It doesn't make sense for me to stay around that long. Besides I have been doing this since the RIPE meeting in Rhodes, that's a long time ago, I don't remember how long that was. In any case, too long.

So, that's the procedure we have in this Working Group to change the Chairpersons, it says, we will be accepting candidates up until the day before the Working Group meeting, in this case I am happy to report we did get one volunteer. I go news, standing there. He is the only volunteer. As you see in the previous point, he is already bringing on his own ideas of how to evolve us, so I think he will be an excellent candidate. So I would like the Working Group to just confirm this, which we will do by just asking if anyone has anything against, and if not then we'll proceed. No one, perfect. Thank you, thank you, Ignas, for standing and you will now be the new Routing Working Group Chair.


IGNAS BAGDONAS: Thanks a lot for that. Just to give a little bit of a clarification, the context of my name coming forward.

So what I was previously explaining, that was mostly trying to find an excuse for doing that. The real reason was that I was peacefully walking through the hallway and somebody approached me and it was stated that if you don't consider putting up your name forward, there will be far reaching consequences. I really had no option.

JOAO DAMAS: Sometimes you have to do things the way you have to do things.

Okay. So, now we actually go to the first presentation. I believe that will be Geoff's. Sorry, it's Juan.

JUAN BRENES: Hello everyone. I am Juan Brenes, let me start with a brief introduction about myself. I have been a Ph.D. student for three years now. I have covered the topics of network function SDN and most recently BGP also. And two months ago I started working for a company called out owes, I mainly work ‑‑ here I am fellow, so I want to thank them before I start opportunity of presenting my work here.

And going back to the talk. I'll be presenting power prefixes privitization for smarter BGP reconvergence. This is a paper we submitted a couple of moths ago with the collaboration of... and Crystal....

The title of the presentation is a little bit long, but I hope you understand that I'm going to talk about the BGP reconvergence, which for me is a still big issue of BGP nowadays and the main questions I will try to answer today is if we can do anything to improve BGP convergence fanned we can do that in a manner that we don't have to introduce any many changes in the protocol. And of course both answers are yes because otherwise I wouldn't be speaking here. What he propose is a very simple but yet effective change over BGP, that has a great impact on the traffic losses or the impact of the BGP failures, and we don't require any change in the protocol so it can be deployed incrementally, it can work perfectly with all the solutions we have a proposal for.

Every time I try to explain what I work on to my colleagues and even to the networking people, they ask me why do you want to change something that already works? And my answer is of course money. We are not deploying BGP in our houses we are deploying BGP in commercial environments and this means that whenever BGP fails, someone is losing a lot of money. The CDNs lose money because they cannot get traffic. ISPs also, we are going to have services that rely on Internet that are going to have latency requirement and so every time BGP fails it's a problem for someone.

Of course we were not the first ones and we won't be the last ones to work on this. There has been a lot of previous research but most of the solutions that I have seen so far are either based on diminishing the time that it takes to BGP to converge or the amount of information that is is exchanged.

Usually, the solutions require changes in the protocol. So they can only be deployed in closed environments and I want to especially mention it two solution that is because I got some feedback in the mailing list and this is a prefix in the regions and out path. This is a great solution, the thing is, cannot use pic when you use the next hop self practice and you need to rely on the IGP protocol in order to update the pointers of the solutions.

We follow a completely different approach, we are not going to tackle either the time it takes BGP to converge or the amount of information that it exchanges. We just base or approach in three observations which are completely compatible with any solution that you can think of. If you want to use power prioritisation or BGP with out path it's fine. If you want to use them with PIC, it's great and you can even use it without any IGP protocol at all.

The first observation that we did is that one single BGP event can affect a huge amount of prefixes. Last time I checked, the BGP table for IPv4 was of about 500 thousand prefixes. And the IPv6 is still a little bit better because it's 50,000 prefixes. But, this is not going to get any better. In fact, today I checked the agenda for this meeting, I saw that the next presentation is called using more specific advertisements in BGP and this is greatly impacting the amount of information that we have in the BGP tables.

The second point is mostly a consequence of the first one. We have a huge amount of information that we need to process and that can not be done at the same time. It's going to take a while between the first update and the last update. We did some experiments with Quagga and in the very best scenario it takes 15 seconds between the first update and the last update for the IPv4 table. So, this is a time cut that is not ‑‑ that you have to consider every time.

The third observation is mainly an empirical observation. Not every prefix has the same meaning for the AS operation. In fact, it's highly accepted that the traffic in the Internet has a powerless solution of prefixes. And this is going to get even worse with the CDNs and everything.

So, our solution is quite simple. We propose to decrease the impact of the BGP failures just by sending the most important information first. It's not something new. All routing protocols already have these kinds of solutions. Let's be smart and let's try to standardise this. We were just trying to standardise a way of providing the routers at least which determines the order that the router has to follow when issuing the updates. Right now, what they do is they follow an alphabetical order which has no relation at all with the relevance that the prefix has for us, so, I think it's a great opportunity if we can come up with a standard to determine this order. ISIS already have these kinds of mechanisms, OSPF, also has it, I think it's a great opportunity to do it in BGP also. It's giving back control of what is happening in the network operators.

Okay, this is the first one. The first part of my presentation. The second part I want to show you some results that I obtained just by using this mechanism over real traffic traceroutes.

So, what we did is we just talked with an ISP and they were kind enough to provide us with two months traces with their link from transit and they also provided WSIS regular BGP tables and what we did is okay, we said we are going to automatically generate prefix rank spacing on the amount of traffic towards the destination. What we wanted to check is if we could use those ranks for a long time and for how long time can we expect to have benefits just by using this automatically generated list. And we wanted to check if the time of taking this traffic samples could affect the benefits that were received by using BGP, and we wanted to check also if sampling the traffic would have any effect on this.

We developed a really easy way of measuring the benefits of PPP, which is we calculated the traffic losses that we have using the alphabetical order which is the one used by the routers today. We calculated the traffic losses which we have by using PPP and we used the ratioey between these two numbers to more or less measure the benefits that we have.

We did that for over three weeks, and we found that in the first days, the traffic losses came in diminished almost to 1% of the traffic that originally I lost. I think this is operationally great. And even after three weeks, the traffic losses can be decreased to only 10%. Okay, we did this with 24 hours of traffic unsampled, which is the ‑ a lot of resources and not easy to do.

So what we did next was try to sample the traffic, at a low rate, which is 100 packets per second. And what we saw is, okay, we have a little bit worse result, but it's going to be much better than the current situation. Even after three weeks, we still have the same boundaries. We only lose 10% of the traffic that the traffic route generally does.

In the article with a lot of effort in order to, mathematically prove that sampling the traffic and having lists that determine the order of the updates is a really good idea and it's mathematically beneficial. So, we developed a mathematical model to bound the minimal benefits that we have. The results are marked with the asterisk in boxes, those are the real results. So as you see the mathematical mode works really good. And even for very low traffic sampling rates, we get really good results, less than 10% of the traffic is lost. And of course the best cases is sampled the traffic but okay this is going to be better for another operator.

This, we wanted to do, see if this could be deployed in practice so, what we did is we modified the Quagga and we deployed two different configurations. One, a full mesh topology and the other ones a route reflector topology. In the first case, we have going to break ‑‑ generated a failure between router 1 and the AS B and we saw what happened in router 2. And the second case, we again we broke the link between the router 1 and AS B and we saw what happened in the route reflectors. What we found is that PPP works even better than we expected because we did some assumptions on them all, if you want IX explain what are the differences, and we even have better results in practice. So, even sampling at 10 packets per second in the route reflector topology, we only get 1% of traffic losses, and in the full mesh topology, we also have 1% of traffic losses, which for me, is great.

So, the conclusion. We can do really much better without having to change the protocol itself. We can deploy this incrementally. We can deploy this whenever we want, wherever we have BGP, we can use PPP and it's feasible to automatically generate the ranks using the amount of traffic. It's going to be a great in terms of traffic losses.

So, now, I will let you ask questions and then I have some questions for you.

JOAO DAMAS: Okay, so question time.

AUDIENCE SPEAKER: Peter Hessler. Have you taken a look at how this algorithm would be applied if you were to mark a so‑called low packet per second but considered extremely valuable AS neighbour or prefix?

JUAN BRENES: No, I didn't check that yet. What we found is that the traffic is mostly regular during the day so the prefix ranks do not change that much. So I guess in than sense we can say okay, this works for a long time, it doesn't have any, I don't know, it doesn't ‑‑ it's not affected by the time that you do the measurements or with the time you do the ranks.

AUDIENCE SPEAKER: Geoff Huston, APNIC. I think this works relies on a common misimplementation of BGP. The theory is with the emry timers that when you have an update to send to your peer, you place a pointer to that prefix in the queue and you wait for a per‑peer timer to expire, and at that point, you drain the queue and send all the updates to your peer. So that you find that you effectively send a burst of update every near 20 seconds, plus or minus 3 seconds, which is the way it works. But that was a Ciscoism that's not in the standard because it is certainly possible to put the timer on the prefix than on the peer.

JUAN BRENES: That's true.

GEOFF HUSTON: As soon as you do that, when you ever got something to say to your neighbour, you say it. Modular the fact that you apply the Emry timer per prefix, in which case there is no prioritisation any more. Once a prefix is due for an update, it sends an update. And so what you are really talking about is altering the behaviour of the misimplementation of the Emry timer in the wait queue commonly used or abused by Cisco, and if you actually did something different, i.e. per prefix timers and spent a bit more time in memory, you would get actually better outcomes anyway. Thanks.

JUAN BRENES: We know Cisco and Juniper does different things related to Emry. We have an experiment we want to analyse how these affect the Internet itself. But, what we found is that it takes a lot of time if you have to analyse 500,000 prefixes, that's not ‑‑ it's not completely related to Emry. Okay, sometime it's been affected by Emry measurement but it's not the whole thing. At least from my understanding, we can discuss that off line if you want.

So, my questions are, if you don't have anything, is it possible to follow with this? Is it possible to try to push an RFC? Do we need to do a draft? Any suggestions would be useful

RUDIGER VOLK: I'll get back to my other remarks before. Well, okay, if you have something that you want to offer in the IETF, just do a draft. You don't have to find supporting groups to submit an Internet draft.

Well, okay, 27 years ago ‑‑ no, much more actually ‑‑ I went to my first IETF and actually I did phone the secretariat in advance and ask whether there was any permission required. No, there isn't. It wasn't back then and it hasn't changed.

JOAO DAMAS: Rudiger, I don't think he is asking for permission.

JUAN BRENES: I'm just asking if you think it's useful to push this kind of approach to IETF?

RUDIGER VOLK: Well, that kind of brings me to the remarks that actually drove me to the mic. To me, it appears we have not really heard a lot about how you prioritise the prefixes, and what I did hear was you were making a statement, which I rephrase slightly. You were saying every BGP failure creates a problem, and that is entirely false. The kind of there are two statements. One is, the Internet is designed and operated as a largely resilient infrastructure.

The other thing that I usually state when people complain to me, is that I'm telling people, well, okay, the Internet is broken all the time by design. And the two go together.

So, kind of, you are starting ‑‑ it seems to me, to some extent, from premises that may not be completely true and to the point. And yes, I think your statement was every time BGP fails, money is lost. And that is completely wrong and it actually points to something where Peter was asking, kind of the consequences of some type of BGP failure can be very different and the value, if you are talking about money, of prefixes that are reachable or are getting blackholed during a convergence process or so can be extremely different.

So, the question of what is your algorithm or what is your metric to define what is important and what is pushed is extremely interesting. And then, of course, a very fine analysis of, well, okay, what types of failures and consequences of failures are to be handled of course also has to get into the process.

JUAN BRENES: So let me first answer you the question about the algorithm. We are not pushing for any algorithm to run the prefixes. We just thought it would be a good idea to use the traffic as a way of ranking the prefixes, but it's not what we ‑‑ we are not taking any decisions about that. So, what we really wanted is to, or the main argument is that I think it would be useful to have a way of establishing the order of the BGP update, which can be used by network operators to determine what happens in their networks. I think there are routing protocols already have mechanisms to do so, I think it's going to be useful to use it in BGP. If you then want to use the traffic as a way of determining this order, okay, what happens is that, as I come from the Akamai provider, so I need to show you some results in order to see this works. But, I could just end my presentation saying okay, we need to have some sort of mechanism, or I think it's a good idea to have a mechanism to say actually control the way the router behaves.

The second part of your question, yes, you are probably right, the Internet is assigned to failure all the time. May I should have been a little bit more polite when saying every time BGP fails we are losing money. But, failures happen and sometimes big failures happen and we have to make them ‑‑ we have to make the most out of the routing protocol because otherwise you might lose some money eventually, okay.

AUDIENCE SPEAKER: Peter Hessler. As we saw in Greg's talks earlier in the Plenary, these micro blackholes are concerned to quite a few operators. The grace he will shut down community and the involuntary BGP session calling techniques that have recently been published are trying to minimise the impacts of these temporary blackholes and that this research helps us show how to change our implementations to minimise those blackholes for what our networks considers to be an important prefix. And so I definitely would like to see more discussion, especially with implementations. The IETF Working Group is probably a good place for this.

So, I would suggest sending in a draft of both of what your algorithm is, how to use it, how, etc., etc., etc.

And then to just quickly answer your second one is, for example on my network, the company I work at, we don't send a lot of traffic to, for example, Amazon S3 but our customers consider it extremely important, so we would like some way to mark certain types of prefixes as, you know, "converge fast," that would help with any micro blackholes from our perspective as well.

JUAN BRENES: Someone asked this on the mailing list and what I said is okay, what we are doing, while we were doing this research we said okay, we are going to focus on the traffic because it's the thing that we can measure. But of course if you want to put your DNS at the top of the list, that is completely perfect. If you want to put your, I don't know, your mail servers, it's perfect. We are not ‑‑ as I said we are not pushing for any kind of implementation. So if you say you are inter sell and at one to put some servers that are not related to traffic, for me it's great.

PETER HESSLER: It's nor a generic algorithm, this is definitely one of the better methods.

JUAN BRENES: Okay. Thank you.

GEOFF HUSTON: I have a second comment. I actually don't think it's one of the better methods. I actually think it's not a very good idea at all. There is two kind of issues you are trying to recover from in BGP. One is the norm at cut and thrust of updates. Your session is up and you are sending across reachability change information. If you were ‑‑ if you were diligent in implementing BGP you would do per prefix Emry timers. There is no prioritization at that point. When you get an update you send it on immediately. If you you really want to get convergance faster, drop the Emry time. Why is it 30 seconds? Because it's 30 seconds. There is no science behind that number. It was just 30 seconds. Has been forever. If you want to play with a shorter number, Juniper made it zero. The second issue is one of start‑up.

In other words you have come down, you are coming back. If both of you were dead, you and your peer and you're knew, you have got to transfer a full route table if one of you was alive the and you have just died, may be the other end would try to keep hold of that session state and what you told it for a period of time, it's just memory. And what you really want to do when you come up is say Hi, it's me again, remember what I told you last time, that stills holds, let me resume from where I was. That's about as fast as you can possibly get. This whole idea of prioritising, telling the neighbour what the neighbour already knew is kind of backwards. You knew that. You know, you are just correcting annum knees ACK problem that the other end if it chose to could just remember it anyway. I kind of this this is warped and twisted in terms of what you are trying to do. What you really should be doing is taking advantage of the fact that one ended failure doesn't mean failure of information at the other end. Use that information and get the other end to hold it and then come back in live and reassemble from that known state.

JUAN BRENES: Just, when we did the experiments and we found that the Quagga takes 15 seconds in the best case to, between the first hop update and the last update, we played with the Emry timer, so, we changed the topology, we changed the timer and the results in the best case, which is with the Emry at zero, as Geoff said, were 15 seconds. So actually, maybe I'm wrong, I might be wrong, but I think this is related to the processing of the amount of prefixes that we have, but I agree, as ‑‑ I can check that for the future, and in fact, we are doing some experiments to determine the Emry in the wild, I was team looking forward to present the experiment, a lightning talk next, so...

JOAO DAMAS: I think Geoff was probably a bit ‑‑ before assuming the state was lost, check whether the state was lost, if it's still there that's the fastest way anyway.

So thank you very much.


Next one up, if I don't mess this around, is you, Geoff.

GEOFF HUSTON: How long have I got? 15 minutes. Whatever, I'll go fast.

It's always commonly said that the BGP is full of needless pollution and that if we all did our job better and stopped advertising more specifics globally, BGP would be so much faster, so much smaller and so much more brilliant, what we have got to do is stop people advertising more specifics because they get in the way and that's horrible. I thought I'd look at that and apply some science to that allegation, and actually understand how bad the problem is with more specifics or not.

The first kind of question is what is a more specific in case you have been sleeping for the last, you know, two or three decades?

A more specific is where you have this covering aggregate like and you decide to announce a more refined, a more specific announcement simultaneously, you know, The inside one is more specific. Why do you do it? Good question, and as far as I can see there are three generic classifications of more specifics. I'm your customer, you have a /8, aren't you great. And I want to take my little piece of your address space and announce it independently of you, so that my traffic doesn't come through you necessarily, it might come to me through other paths as well.

So, that idea is what we call hole‑punching; that all the traffic still goes to you except for my traffic, which might come through entirely different paths to come to me. So, I'm announcing the more specific with the different origin AS. And its route paths may differ as well but that doesn't matter it's a different origin AS. Here is an example. If you are one of these people you know you are doing it. This is a real live dump. I don't know where they are, it doesn't really matter; it's an example of hole‑punching.

However, that's not really that common.

The other thing that's sort of more common is traffic engineering. Not everyone, if you will, has enough capacity on all of their links to carry their entirety of load. So when you get different links to different upstreams you might want to even the traffic across all of them so that you sort of not over‑purchasing capacity at any point. And this is pretty common. So what you see in an example of here is dividing up that /8 into two halves, a /9 being advertised through one transit, and a /9 through another. Same origin AS, but different AS paths. There is an example. 1.37 does it this way. Where they announce more specifics through a different next hop. So they are trying to sort of shift some of their traffic through a different ingress path to try and even out the load, I would assume. If you are 4775, well ‑‑ that's my guess as to what they are doing. The next one has me a bit fooled but it's quite common really. It sort of gets close to senseless routing vandalism where you do is you take the aggregate and you divide it up into a comprehensive set of more specifics that have exactly the same origin AS and exactly the same prepended path. So, you are not doing traffic engineering, it's the same traffic. You are not doing any kind of hole‑punching, it's the same origin AS. It's just, well, I don't know. You are just announcing it because you just can., wherever they might be on this planet, there is a good example, it's one of zillions.

So, those are the three types. How many of them are there? In v4, 53% of what you carry in your routing table which a few days ago was 685,000 entries, 53% of them are more specifics of one kind or another. We advertised 2.86 billion individual address spaces in v4 of which 1.04 billion are actually more specifics. So, approximately 30% of the address table is covered that way. You know, over one half of the table doesn't give you new data. It refines existing reachability, not new reachability. So, it's a refinement in some ways.

So the next question is, well, that sounds pretty weird, half of your time in your routing table doing more specifics, is that always the case? This is a graph for the last ten years, those two lines the number of total advertisements in the routing table, the number of more specifics. I can doing it in v6 as well, there is a similar pattern going on there. A bit hard to interpret. So let's talk about percentages.

In v4, since you know, forever, half of the routing table has been more specifics. I don't know how you do this. I don't know how a bunch of people all over the planet, all 50,000 of you, collude together in rooms I have never been able to get into, on mailing lists that obviously bar me and you go will I do more specifics? No, wait your turn I'm going to do for specifics today, because that's so stable at 50% it's totally uncanny. I bet you guys are talking to each other and just not letting me in on the conversation. You are also not letting the v6 operators in on the conversation because they haven't got the clue yet but they are getting there. It's still growing. The number of more specifics is slowly reaching 50%. V6 will be real when it finally gets there.

So, I said before, there were kind of three types. The hole‑punching, the traffic engineering and the over lays and I have colour coded them. That's a bit hard to read that way, so again let's look at percentages. And let me just sort of draw some pictures there. What's fashionable these days in the routing circles is senseless routing vandalism. So, these over lays are increasing in percentage terms while the hole hunching, I have got your address, is actually decreasing. The RIRs and their address allocations, all the holders of addresses themselves are less willing to have folk hole‑punch. So, hole‑punching is decreasing in terms of its use. Traffic engineering, the green, kind of constant and like I said, senseless routing vandalism is kind of the fashionable thing to do.

So, that's just what I said. You can read it too.

What about the address span? Is more and more traffic or addresses being covered which more specifics or not? In v4 you can kind of track this that while it's half the address table it's it's certainly not half of the addresses. That green line is relatively low. In v6, it's really really really low because I had to resort to a log scale to get that yellow line coming off the bottom. So, in v6, it's around 2.5% of the total v6 address span is more specifics. Whereas in v4, it's slowly climbing and it's climbing to just around 30% these days, on average. So, around one third of the address space is done with more specifics. 27% versus 2. So again I can break this into various types. How much space? It's mostly over lays. So, most of the time, most of these specific addresses are being done with overlays which convey no new global routing information. Nothing. Same addresses, same origin, same path by the time it gets to me. There might be something local going on, but by the time it hits the broader scale, no, I can't see it.

So, it's all about overlays. It's all about overlays. And it's only overlays more recently. It used to be I took some of your addresses and separately advertised it in v4, but that's now no longer the case. Overlays are what you do. But are they polluting? Do they actually change the load on your router? Because, putting something in the routing table is kind of, and if you are got decent caching in your line cards it won't alter that much. Are they causing you endless processing of updates?

So let's actually look at BGP updates. And the real question is, if I'm trying to do traffic engineering, do I constantly try and churn BGP and more specifics? Are these specifics noisier or not than the aggregates they are refining? And while the overlays are certainly more previous prevalent, are overlays noisy? Do you constantly refine these overlays one way or another?

So, I looked at the update count, and then across all the updates over ten years ‑‑ and that's a shit load of updates. It's 100,000 minimum per day over ten years, you do the maths. My poor machine melted but just before it melted it gave back these answers. More specifics generally in v4 are noisier than the aggregate. So, per day, you know, what you see are the more specifics actually giving me more updates than the aggregates on the left. V6, it's the opposite. Now I find that a bit weird, and you should too, it's kind of ‑‑ I'm not sure we actually deliberately use a different BGP configuration in 6 than 4, you don't, certainly not eBGP, so exactly what's going on there? Interesting question. Wish I knew the answer.

So I think that's just the text of the graph. The relative count of more specifics is lower than the aggregate in v6, but that's absolute counts. And like I said, there are very, very few more specifics in v6. What about in terms of relative counts? The proportionality ‑‑ I thought I had scrawled some lines on there. What you actually see is in v4, the green is slightly above the purple so more specifics have a slightly higher probability of being an update than v4. In v6 it's still a pretty noisy line. More specifics, aggregates... more specifics aren't that noisier. So on average the v4 more specifics are noisier than v6? Not really.

So the last sort of question to look at is if I break this up into hole‑punching, traffic engineering and senseless routing vandalism, which one wins? The green, which is, I recall, senseless routing vandalism, tends to dominate. So, these updates are actually updates that don't change the AS or path. But there is an awful lot of attributes that seem to churn. And I find this relatively strange. So in v4, there is this obvious picture that these kind of overlays tend to come from somewhat unstable BGP configurations. V6 is just noisy. In routing terms, day by day, days are different to other days. And that the more specifics also tend to be quite noisy. The overlays tend to be higher than the others, but it's by no means a common state.

So, what we're seeing here is the overlays in v4 show the highest level in terms of updates, whereas in v6, things are subtly different.

But, you know, there is a thing called brown yen motion, that in a glass of water, all the atoms kind of bumble around at the same speed. There is not a small group of atoms going really fast and all the rest going really slow. And the mace take in BGP is thinking the routing system is uniform, that an update is equally likely from anywhere on the planet, any part of BGP. But if you think that you are incredibly wrong. 40% of all the updates in v4 come from ‑‑ sorry, yes, 40% of all the updates come from 1% of the prefixes. That's what this is showing as a cumulative distribution. That in actual fact a very small number of prefixes in BGP are incredibly noisy, half of the prefixes, half of them generate less than 10% of the updates. In v6, the noisiest prefixes, that top 1%, generate 60% of all the updates. So, 60% of all the updates come from just 1% of those prefixes. So that what I'm looking at is actually not BGP. I'm looking at a very small subsection of BGP where those prefixes are noisy. So, when I look at it like that, and try and understand who is going unstable, I get a subtly different view.

In that view, when I take into account the skew, traffic engineering is twice as likely to be unstable than anything else, which is kind of intuitively right, that when you do an advertisement to try and change traffic, then you are likely to do a refinement of that advertisement to change the traffic again if you are still getting traffic in balance. And you might have put that into some automated system so that the changes will be coming as often as you need without operator intervention. And so the intuition about what's going on kind of matches what we observe, that overlays and hole‑punching is relatively static. It's what you set up. Whereas that kind of traffic engineering, dynamic response, is kind of what you'd expect the traffic engineering will cause a high degree of instability.

So, that's where we are, that the route prefixes and the hole‑punching tend to, in relative terms, in both v4 and v6, tend to be the more prevalent form of update.

So, what do you conclude here? Well, it's not that different. They are not a vastly different routing load. It's not that these more specifics that 300, 400 times the loads of the aggregates most of the time. But v6 is an anomalous network, smaller number of prefixes but the ones that are unstable are incredibly unstable. 100 times for instability events in v6 than their comparable peers in v4. I have no idea why. I have no idea what we're doing that makes v6 so incredibly unstable.

So the next thought experiment. What have everyone listened? Good God, what a thought! Oh, I must get rid of all of my overlay prefixes because actually it's sensible routing vandalism and today I'm going to be a good Internet citizen and get rid of all those stupid overlay prefixes. What have you removed them? Well, your FIB might get a few more, what, two years, two and a half years, of expansion space. Actually there is something weird in that v4 picture. That spans from 2007 until a few days ago. Look at that purple line. When did we run out of new addresses? And why don't I see it in that graph? Really good question. For some reason BGP is defying gravity and we keep on inventing more and more prefixes to advertise irrespective of the fact that the registries aren't really giving out that many more addresses at the same rate they used to be. You will buy around two and a bit years if you got rid of those overlays in v4. In v6 you'll buy about one year of time, because there is only about 10,000 of them out of the 40,000 of them.

What about updates? What about updates? The update count in v4, the daily update count. Just under a million or so. What if we remove the overlays? The purple line versus the green line? You'll only get extremely marginal benefit. And even in v6, the overlays aren't really the problem. If you removed all of the updates that you could attribute to overlay prefixes, the load on your router would be about the same.

So, the size, you'll see a size drop by around 30%, but dynamic instability, they are not really that important.

So, you know, there is no doubt that more specifics do add to the size and the update load. But at the same time, we can't do traffic engineering outside of BGP. You just can't if you want to do inter AS traffic engineering. In some ways the traffic engineering prefixes, and even the hole‑punching prefixes are kind of part and parcel of the environment we live in and it would be senseless to say to you to stop it because that's the way you do routing and that's okay. It's only the overlays you might think maybe you shouldn't but they don't really add that much of an incremental cost to the grand scheme of them. They are becoming more prevalent and that's kind of a FIB issue, but in terms of the total routing load... no.

Why is v6 more unstable? I have no idea. If you have an idea, please tell me. Because it's just a whacko situation.

And I think that was it so any questions?

JOAO DAMAS: Thank you, Geoff. Any questions?

AUDIENCE SPEAKER: Owen DeLong, as to Y v4 is defying gravity I would propose that the hierarchy are creating plenty of new prefixes by splitting the owed ones through the transfer process, that's why you haven't seen any slow down in the BGP growth rate even though you have seen a slow down in the handout of fresh addresses.

GEOFF HUSTON: There is a presentation that I have given at times before when I analyse all the new prefixes that appear each year and try and figure out if they are in the transfer registries or some other issue. And it's certainly true, back in 2010, most of the new prefixes each year from allocated sort of six months before. And it's certainly true today most of the new prefixes were allocated somewhere back in the midsts of time, but only 10% of them are in the transfer logs. Either the transfer logs aren't capturing all of transfers or I have no idea. Whatever, someone shouted reserves. I have no idea. But certainly it's true, own, there is a certain amount of churn going on, but the overt transfer legacy don't describe what's going into the routing table in its completeness.

AUDIENCE SPEAKER: Then in terms of the reason that v6 is so much more unstable, I'm going to guess that a lot of unstable prefixes you are seeing are probably 48s. And they are probably a large quantity of home users messing with tunnels and other people experimenting with BGP and v6 and turning it on and doing their little lab thing and turning it off when they go home, so it doesn't mess up their production network because they are afraid.

GEOFF HUSTON: It's a common sport in v6 to blame everything that's bad on Teredo.

AUDIENCE SPEAKER: I wasn't doing that. I was thinking more of Tunnel Broker.

GEOFF HUSTON: I'll BGDp and blame Teredo. You could well be right. I need to look at it from that angle and look at the prefix size, good suggestion.

AUDIENCE SPEAKER: Then on why you see more prefix instability of, or more covered aggregates in ‑‑ or sorry, more disaggregation in the category 3 announcements in v4 than you do in v6, although v6 is as you have said catching up, is probably because a high percentage of that comes from end sites in my observation, and there are a lot more end sites with more than one prefix in v4 than there are in v6.

GEOFF HUSTON: I heard. I heard there is this mythology out there that if I announce for specifics I can't get hijacked. This is about the same as if I hit my hand on the horn of my car and beep loudly and continuously, I'll never have a crash. Now, all of India might believe the second and they are wrong. And the first thing if I advertise a more specific, no one else will advertise the same more specific anywhere on the planet, that's wrong too. So I suspect, yes, there is a certain amount of if I do this nothing bad will happen to me, which is kind of wrong.

AUDIENCE SPEAKER: Peter Hessler, with my corporate hat on. So Hostserver. We have noticed a lot of instability in our v6 routing updates, and the RIPE BGPlay for our AS is essentially unusable unless I filter out our picks announcements. I am the one who is doing those announcements. I announce only the allocation and it's static. I haven't changed it in months. But, RIPE shows that some of our neighbours are just flip flopping every three minutes, every five minutes, and that could indicate a problem, and I was wondering if you were able to look and see if you see a flip flop and filter those out and see if v6 is stable or instable and if it's a certain number of players involved in that.

GEOFF HUSTON: I did some work sometime ago about playing updates through a big cache and finding that with everyday of updates most of the updates are things you already heard about a few seconds or even a few minutes ago, that a lot of updates just repeat information and the new information load you need a microscope to find it. Which, sort of again matches intuition and I think we could find the behaviour you looking for almost around the world but then comes the issue of why v6 and why not 4? And we need to look at that? It just isn't ‑‑ it's not behaving as it should, is it?

AUDIENCE SPEAKER: Correct, and I am announcing our v4 allocations to the same neighbours and they are not ‑‑


AUDIENCE SPEAKER: So what are you people doing with v6?

GEOFF HUSTON: We switched on the I hate v6 on the router. I don't know either.

AUDIENCE SPEAKER: If there is one I want to know where it is so I can turn it on too.

JOAO DAMAS: Can I ask everyone to keep it short.

AUDIENCE SPEAKER: Jen Linkova. From my operational experience, broken things get fixed when someone complains. So, I wonder if you look if you actually see any users in those unstable prefixes?

GEOFF HUSTON: I cannot see users. I only see the routing table. I do not run an operational network. I don't see traffic.

AUDIENCE SPEAKER: I mean, in your other experiments for example, if you see any users, because I suspect that because of what we have, 20% of v6 currently, right, so 20% of users might see, might complain about the v6 network being unstable, as a rule, so you are probably just seeing if there is a broken configuration, nobody noticed?

GEOFF HUSTON: The one thing about v6 is that v4 makes it all better. And happy eyeballs is a testament to that so it could well be that users could have complained but the problem went away because when it happened again it all became better. I can't see users, I can't see traffic, I just see routing, so, nice idea, but personally I can't do it, maybe you can.

AUDIENCE SPEAKER: Second question. So, I suspect, I wonder what the probability is that some of the prefixes you see as Type III, actually are Type II, Type I, you just do not see another route.

GEOFF HUSTON: You are quite right because again BGP masks localised but the real issue is why in our tools of BGP does local policy become global noise? Why don't the more specifics stop doing what they are doing when they cease to be effective? And we have never yet worked out the idea to define ‑‑ I want my routing, the more specifics to propagate around this part of the world and then stop and if someone could give me a language to do that that might be a good thing, go communities.

AUDIENCE SPEAKER: Ben. In my daily work at DE‑CIX I am also confronted with this type of thing and thanks for making it a topic. I would like to add two possible reasons what I believe are true. That we have more specifics, so first, people sometimes think it's cool to have a larger prefix space, and it increases their chance to get some larger peerings. So, maybe we can help these guys and they think that it's more cool to have just super networks in the routing and not many many prefixes, so, this is statement number one.

Statement number two is, I see that people often have just no clue about what they are doing, so, they believe that doing /24s is needed, so maybe we can help these guys to explain to them that this is not needed to do this kind of fragmenttation and here I would also like to point on the upstreams because maybe it's not always needed to work with less or equal statements to allow these kind of prefixes passing through, and if the upstreams ask the customers, hey, bring it down to the super prefix, then this maybe would also fix them.

GEOFF HUSTON: It's always hard to say to a fee paying customer, I'm going to refine what you are telling me, and a lot of ISPs kind of go well they are paying me, I'll do what they said. It gets hard.

The other thing is that we can identify the folk who do this, because they are the folk who believe announcing /24s makes them invulnerable to attack. They are the same people when they jump into their car to drive home put their hand on the horn because when the horn is going you'll never have an accident, will you?

AUDIENCE SPEAKER: Rudiger Volk, Deutsche Telekom. Geoff, your analysis of routing vandalism, this time includes only vandalism with safety belts on, meaning well, okay, there is actually the aggregate announced. I think in earlier analysis, you also looked at the vandalism without the safety belt. I guess you still have the statistics, how small is that?

GEOFF HUSTON: Oh yes. Oh yes, you are advertising you know four /24s, why aren't you doing the 22? All the 24s look the same, just give us the ACK gate and go away. I published all that on the CIDR report, but I didn't pump it through the update engine, because this already took me about two months of CPU on the biggest CPU set I could find. Adding another test would simply extend it to years. I couldn't do the update analysis and it's a good question, and if I return to this work, I'll make sure I do it. Thank you

RUDIGER VOLK: Let me just point out, in the databases, I am seeing really threatening examples of vandalism in v6, like, well, okay, why don't we publish route objects for about 48 Ks of /48s out of our 32.

GEOFF HUSTON: Because, you know, more is better. Peter, you seem to have a response that's urgent.

PETER HESSLER: I have a response to that, and I have created route objects for source 24s in our IPv4 in our IPv6 is I am using I forget what it is, but there is Anna attribute saying I am allowed to announce 48s, it's and it's very specifically for DDoS mitigation, I nominally only announce the allocation but if I'm under attack I need the route objects to exist, so my neighbours and other entities will accept them.

GEOFF HUSTON: Are we doing the right thing, future agenda item someone, to do DDoS response by changing routing prefixes which forces naturally to advertise more and more specifics so that the response matches the DDoS target? I can understand you're doing it because you have no other way, you think. But is the routing system the place to apply the band aid and is suppression of a route the only way we have got? It's a really good question. And I wish we'd talk about this more, because your FIB can't take a billion entries. It just can't. Where we are is okay, but where this is going is not okay. And if it's DDoS driving us there, which is part of the reason, it's a really good motivation, but I'm not sure we trouble understand how we want to respond. Thank you.


JOAO DAMAS: We have three further short presentations, the first of which is by Ignas.

IGNAS BAGDONAS: Hello again. And a quick update on trying to address and fix one of the practical problems, and this is an indication of what is happening in one particular IETF Working Group.

So, pseudowire. A pseudowire is a tunnel mechanism which allows you to carry any type of traffic encapsulateed in some other lower layer, typically that's MPLS. Pseudowires have been around for a while, the design of those reach around 2,000, the year thousand, and in particular, here we are discussing the ethernet of the wires, it's a very conconstruct. You take an ethernet frame, attach a few MPLS labels on top of that and send it out. It just works. On the other side you receive that tunnelled ethernet frame.

So, this mechanism is widely deployed and it works. Now, one of the mechanisms that pseudowire has is called a control ward, it's a simple short header which sits between the pay load and the table stack and which can carry the service information. At the time when pseudowires were being standardised, the equipment that was at the time and we're talking roughly 15 years back, was not able to process the control without a substantial penalty to the performance. And at the same time, the Mac addresses allocated at the time were not starting with 406 and the registration authority wasfully not allocating such blocks. So, therefore, it was decided that due to these practical constraints, it is safe to use pseudowires in a simple form without any control word, just put the ethernet address and send it out.

Things have changed since a little bit. One of the main aspects is that now we have ethernet addresses allocated which start from 406, and also the equipment itself has evolved and today doing lookups which involve looking up multiple layers of heathers is no longer a problem. And some smart network elements can do actually even deeper lookups and do more processing than that. What is the end result of all of this? So, I get my new shiny router and apparently for no obvious reason that router starts to re‑order flows or even in a worse case starts to introduce random traffic drops. So, what is the reason for that?

If we look at the, how things look on the wire, how things are encapsulated on the wire, there is no unambiguous way to distinguish an ethernet packet which starts with 406 encapsulated in a pseudowire control world from a normal IPv4 or IPv6 packet encapsulated in MPLS. The first nibble one is 406, therefore the router looks up beyond the label stack, finds 406 and thinks okay this must be IPv4 or IPv6 packet. Therefore, I will try to calculate, try to find out the source destination addresses, ports and other things, and put it into the appropriate queue. Or even if I'm more smarter I will try to calculate the checks and the pay load and what happens? All of this is done on a random data because the packet is not IPv4, IPv6, it's just a plain ethernet pay load. The end result is that this mechanism introduces either misqueue‑ing of the traffic or the worse case drops.

How control word can solve this? It's a very simple approach. You just inasserted heard in the middle which can never be 406, just the first nibble of the control word is always zero. Therefore, there is no practical way to mix it up with the IPv4, IPv6 encapsulated packet. And this is exactly why pseudowires, why the control word in pseudowires have been designed. So, from a contextual point it's a very simple solution.

There are side effects. You need to carry additional label, that's additional, that if your environment has a very tightly controlled MTU, this is not work for you. This might potentially break the clever or not so clever middle boxes which try to pretend they are clever by looking up into the pay load and they will not be able to recognise the encapsulated pseudowire traffic there.

Why does it matter? This is an actual operational problem. There are complaints and complaints show up frequently that for no apparent reason my route is dropping the traffic. This is also coming from IEEE saying that yes we probably made a mistake allocating addresses starting from 406, can you please address something on your protocol site. Pseudowire has a mechanism very widely deployed and mostly universal use. So, therefore, there is a simple document in IETF, trying to address exactly this problem. Whenever you are using pseudowires, check your configurations that you, in fact, are using control word for ethernet pseudowires. This will just increase the quality of your sleep.

That's it. Thank you. Any comments or questions?

JOAO DAMAS: Thank you, Ignas. I won't insist because we are running a bit late but if you want to talk to him you know who he is now.


Okay, next up, Database Working Group Chair is giving us a small report and you have heard the discussion on the list about foreign route objects in the RIPE delegation.

WILLIAM SYLVESTER: I am the database co‑chair for ‑‑ I just wanted to talk to you guys today and let you know about an initiative we have going on in the Database Working Group. We have just recently asked RIPE NCC to look at our proposal that we just approved last week. They are going through a solution review, and here is basically the gist of what it is.

We're proposing to remove the origin from the RIPE database and with that, we're also going to create something called a RIPE non‑off for out of region objects. This is basically working with the aut‑num objects, it's something we have been discussing in the list for over a year. The primary motivation behind this was to align with some of the other regional Internet registry databases and to try and provide some better understanding of how we work with some of these out‑of‑region objects.

So if anybody has any interest, or if there is stuff you are welcome to talk to me or come to the Database Working Group today at 2 p.m. in this room. Otherwise, we open up opportunities for folks to join the mailing list and discuss, and with that I'll take any questions.

JOAO DAMAS: Thank you.

AUDIENCE SPEAKER: Shane Kerr. I work for Oracle, but I'm not speaking for Oracle. You said you were moving origin, you mean you are just moving origin authentication.


RUDIGER VOLK: I understand this is something that was supposed to close an NWI?


RUDIGER VOLK: Where is the actual solution published for the NWI? I see the initial e‑mail, easily findable. And, kind of, I was disabled from closely watching stuff for most of the year, and kind of I didn't see a really clear proposal in the last call.

WILLIAM SYLVESTER: We can take that off‑line and talk about it further.

JOAO DAMAS: Okay. Thank you. Traditionally, this group has cared a lot about routing registry aspects of the RIPE database database, if you still care about that please look at that and provide your input. Last on the agenda, Kevin...

KEVIN MEYNELL: A very short report on the a BoF that was held earlier in the week. This wasn't formally on the agenda, although it was sent out to a number of people invited to come along. So this was actually held in the bar upstairs, we got a few people with, shall we say, the great and the good of the Internet routing community to come and give their comments.

So, what were the objectives of this? So, this essentially was called by the MANRS initiative, some of you may be aware of this. This is a community initiative that's kind of supported by ISOC with the aim of promoting the culture of collaborative responsibility for the stability of the routing system. And this is sort of defining these four concrete actions that network operators should ‑‑ well, are recommended to implement. So, filtering, at this spoofing, coordination and global validation, there's been ‑‑ it's been presented before at RIPE meetings and my colleague has sort of been leading that initiative.

It wasn't really about MANRS. It was really to try to discuss some ideas for how can we measure the health of the Internet routing system. And to get some inputs and ideas behind that.

So, the aim really was to develop some empirical data to strengthen the case for collaborative routing security, you know, what is good, what's bad, what kind of metrics can accurately reflect this?

So, there were about 20 participants at the BoF. Some of the guilty people are in this room, but the summary points:

Well we actually didn't manage to achieve what we set out to achieve at the BoF but it was quite a constructive discussion nonetheless, so really, the people that were there, I think there was pretty strong consensus on this. Really there is not a lot of purpose trying to devise metrics to measure the health of the routing system until we identify why previous and current attempts to address the issue of route leaks and hijacks, general BGP turn, are essentially haven't been successful. So less than 2% of IP prefixes causing 90% of the routing updates. So, the problem is arguably relatively small, okay, in the context of over 750,000, it's 2% is still quite a lot, but it's a relatively small problem in principle.

And also, the problem with any tool or metric that's detecting the route misconfiguration, or hijacking, it's actually to find one that doesn't generate too many false positives.

So, really, whilst you want mechanisms to be able to identify the leaks and make it difficult, these also need to be lightweight enough not to affect the basic functioning and stability of BGP.

Okay, so, what's the way forward? Right. So, this is what the consensus is. This has been circulated. I got some feedback on this. Please don't shoot the messenger. I am just the reporter here.

Okay, so, there was this, I think, a strong consensus here that RPKI is not addressing the problem and there there needs to be this analysis why the mechanism has got such limited deployment currently. It was also felt that BGP SEC which has just become an RFC, that will also not address the problem and it's unlikely to be deployed in the future. Some people will take issues with these statements, we know, like I say this is just what came out of the BoF.

So the feeling was here that the discussion should be removed from the IETF and also those with direct interest in the development of RPKI and BGP SEC. And actually an analysis, a dispassionate analysis taken of the vendor and network operator industry.

So whilst I am working for ISOC, we did not suggest this. This came from the participants, so ISOC was asked whether we can organise a series of stakeholder workshops with vendors and network operators and we need to get the major ones involved in this as well, because ultimately if we are coming up with solutions that aren't acceptable to the major players, then it's not going to go very far. But, also this would involve the RIR communities as well. So, really what are the problems or specific concerns as they see it and how do they think we should go forward with this. So really try to get this postmortem, what things are liked, what things they think would be accepted in the future, do we need to do anything at all. That's also been a suggestion.

So, how we take this forward, we'll go back and discuss this with ISOC and with the people who are in the BoF and this community is welcome to participate as well in this. I don't know how we are going to do this, we'll work out some detail about this. But that's the sort of plan for going forward.

So, that's it.

JOAO DAMAS: Okay. Thank you Kevin. Rudiger.

RUDIGER VOLK: I strongly object to the third bullet. So you do not want to see me in any discussions? I know I am annoying.

KEVIN MEYNELL: You were in the BoF, right, if I remember.

RUDIGER VOLK: Okay, I came late and I did not see and hear that phrasing. And maybe you phrased something that is different from your intentions, but if the phrase is taken as written there, you are saying, well, okay, everybody who is working on that and has some interest in that should be ignored.

JOAO DAMAS: If I may having been there, I think the wording was more along the lines of the IETF was taking a formal academic approach to design this and a little bit detached from operational reality. And that was why it should ‑‑ not only be taking place ‑‑

RUDIGER VOLK: So that translates into we abandon the hope that IETF gets connected to reality that's also a very bad thought.

JOAO DAMAS: That was what was said.

KEVIN MEYNELL: I mean, look, don't get too hung up on the words here. Honestly, I'm happy to ‑‑ well, I'm just the messenger here, but I think the group or whoever is putting this together will you know ‑‑ I think we're happy to take on board other suggestions.

JOAO DAMAS: I think Peter was first and then you.

AUDIENCE SPEAKER: Peter Hessler. I was one of the guilty parties in the BoF and I believe on the third point the intention was not so much to exclude anyone who has ever thought about BGP SEC or RPKI or anyone who has ever been involved, but to remove those who are extremely close to it like the primary authors, because there is the likelihood that emotions and like an at that. Will colour the view and will prevent a dispassion yet analysis. And a dispassionate analysis is the primary reason that we were thinking this should happen. So maybe it was a bad suggestion, but that was an idea that was floated.

AUDIENCE SPEAKER: Tim, the RIPE NCC. I am certainly aware of the deployment issues around BGP Sec but it's important to differentiate from origin validation and BGP SEC. The RPKI, as such, essentially connects keys to resources, you can make attestations about origin validation for example, path security, I think there could be ways that this can be tied in to a RPKI model ‑‑ well essentially, that you can use keys in the RPKI to make other attestations than the ones you find currently in BGP Sec. And I would say that the people have ideas about how to do this than taking that to the IETF, in my view, would not be a bad idea. And there is a CIDR ops Working Group nowadays that has also started to address more of the operational issues.

JOAO DAMAS: Okay. I guess that's ‑‑ okay, I don't know how you are going to write this down in the end.

KEVIN MEYNELL: It's not me that has to do that. I take that on board, the points. It may not have been completely accurately represented. It was just you know, it was circulated in advance, this, to the people that were at the BoF. I don't know if I had Rudiger's correct address, so it's possible he didn't get that.

RUDIGER VOLK: I'm only reading my mail when I get home.

KEVIN MEYNELL: Fair enough. Please don't take this as some sort of set in stone. This is what I felt were the conclusions that came out of this. We can soften the language.

JOAO DAMAS: With minor modifications I think that's what was said. I think Tim's point is okay, but actually Tim's point is part of the reason why this was said out loud. I think an external third‑party that has a more objective look at why it's not going anywhere might be what we need.

Okay. Thank you very much.

With this, we are done with the routing here. Me personally, I am checking out. Good‑bye you all. So long and thanks for all the work. Your new chairs are now Paolo, who continues, and Ignas. See you next time.

(Coffee break)