WEBVTT 00:00.000 --> 00:17.000 Thank you, thank you for having me and also thank you very much for staying until 00:17.000 --> 00:20.200 so late on the last day of fasting. 00:20.200 --> 00:25.600 So this session is called the effective standard setting and I'm sorry this is going 00:25.600 --> 00:33.200 to be a bit rough as I start but you know the question is why this title and the reason 00:33.200 --> 00:37.720 for this title is the current ESO process as well they're not very effective to put it 00:37.720 --> 00:46.280 mildly right there's slow they fail to involve the main experts and the standards they 00:46.280 --> 00:53.320 produce are unshit for purpose more often than not and this could sound like this 00:53.320 --> 00:58.640 gruntled open source community person renting but it's actually not my words it's 00:58.640 --> 01:05.220 literally what the report from the commission based on the you know the ESO's own 01:05.220 --> 01:11.860 research shows right and you know if you want to dig into it you can most of my 01:11.860 --> 01:19.760 comments were based on 0.311 page 19 but the purpose of this talk is not to do 01:19.760 --> 01:27.780 standard organization bashing it's to propose an effective alternative and I have talked 01:27.780 --> 01:32.820 about the reason why the ESO's function the way they do in their history and the 01:32.820 --> 01:36.740 past and I know and I understand the context and I'm you're more than happy to 01:36.740 --> 01:42.360 go look at that talk if you want a sort of more understanding perspective on 01:42.360 --> 01:46.760 the situation but really the purpose of this talk is to try to figure out how 01:46.760 --> 01:51.800 could we do this more effectively and we actually just had the premises of a 01:51.800 --> 01:55.600 demonstration of that was the process that Etsy has put in place for the 01:55.600 --> 01:59.600 vertical standards of the CRA which are clearly going in a direction that 01:59.600 --> 02:06.400 essentially I'm going to be going to be talking about so the good news is actually 02:06.400 --> 02:13.760 effective standard setting is fairly easy and we actually already have a model 02:13.920 --> 02:28.280 for effective open collaboration at scale. Ta da it's just open source and you know 02:28.280 --> 02:31.520 there's sort of pushback I get whenever I say this as oh but Toby you know 02:31.520 --> 02:34.800 like a standard it's very different like it doesn't work that way at all we 02:34.800 --> 02:42.560 cannot do this for lots of reasons reasons reasons but a number of 02:42.560 --> 02:46.920 consortiums whose standards and specifications we all rely on 02:46.920 --> 02:53.480 economic W3C etc have been using techniques like this for over a decade at 02:53.480 --> 02:59.720 this point and it works really well and they publish regularly you know 02:59.720 --> 03:04.920 increasingly rapid pace standards and specs that we all depend on and that 03:04.920 --> 03:15.640 our technology depends on so yeah it works try it right and so how exactly 03:15.640 --> 03:22.080 does it work well essentially as we were seeing before you put specs and as 03:22.080 --> 03:28.320 text files on markdown files in publicly available code repositories and then 03:28.320 --> 03:34.080 you just use issue trackers and four requests to track comments and manage 03:34.080 --> 03:38.880 contributions so you can already see the difference between the Etsy model or 03:38.880 --> 03:43.320 just presented it and the way the open source works is you know with this 03:43.320 --> 03:49.720 model you can actually make proposals right the editor reporter operates 03:49.720 --> 03:53.360 essentially like a maintainer and you want to go into the governance of what 03:53.360 --> 04:01.560 that means in a few seconds and what is really really important is asynchronous 04:01.560 --> 04:09.320 communication right for those of us that were involved in the sense that 04:09.320 --> 04:17.520 like horizontal standard work essentially having to sit through three days of 04:17.520 --> 04:23.280 meeting to be sure that you would not miss the moment where your comment would 04:23.280 --> 04:29.720 be addressed but then still missed it is incredibly painful and it doesn't 04:29.720 --> 04:35.000 make any sense and also cuts out a lot of people that do not have the 04:35.000 --> 04:40.400 luxury to have more organizations willing to pay for them to sit in a 04:40.400 --> 04:45.680 meaning for three days so you know asynchronous communication is really 04:45.680 --> 04:50.640 important and then of course what you need and again that is not rocket science 04:50.640 --> 04:56.800 either or IPR models that allow for this open collaboration model and then 04:56.800 --> 05:03.720 allow you to close down a final version that cannot be that cannot lead to 05:03.720 --> 05:11.440 their derivative work so that people can trust that that standard is the 05:11.440 --> 05:16.080 one that was reviewed and approved by the consortium organization and not a 05:16.080 --> 05:21.560 fork of it that would have a few things modified so again like this isn't 05:21.560 --> 05:29.360 practice like it exists right this model works. What's really important and 05:29.360 --> 05:33.840 what changes from the models that we see in the ESOs is that the drafting 05:33.840 --> 05:40.240 and the review are open right you start from the get go essentially was an 05:40.240 --> 05:46.880 empty page or perhaps an initial contribution and all of the drafting happens 05:46.880 --> 05:52.080 in public and you have you get as a result early feedback and use cases that 05:52.080 --> 05:58.160 feed into the standard work itself and then you get continuous reviewing the 05:58.160 --> 06:06.480 whole drafting phase. So depending on the organizations you have that have 06:06.480 --> 06:09.760 adopted the system and depending on what you need or you can have you can be 06:09.760 --> 06:17.840 very flexible and creative with the governance. As an example some for example what 06:17.840 --> 06:23.480 we G uses a concept called living standards where they're essentially always 06:23.480 --> 06:29.320 contributing to a standard that is never finished. The idea is the standard 06:29.320 --> 06:35.920 reflects the interoperability of the ecosystem and is updated continuously. 06:36.400 --> 06:44.800 Full legal reasons they publish on a regular time a sort of like a version 06:44.800 --> 06:51.560 copy of it to trigger IP our commitments from the broader industry but the 06:51.560 --> 06:57.400 standard itself is just constantly living. On the other hand at the opposite 06:57.400 --> 07:02.200 of the spectrum you can leverage the exact same system for the drafting phase 07:02.200 --> 07:07.480 and then have very formal review process and that's actually used for example 07:07.480 --> 07:11.680 to move standards built in these consortiums to ISO and make them 07:11.680 --> 07:19.360 international standards in the process. You can have very different forms of 07:19.360 --> 07:25.360 consensus mechanisms do. You can empower maintainers of reporters to essentially 07:25.360 --> 07:31.640 be driving the standard and have a good sense of what the consensus is 07:31.640 --> 07:37.600 going to be and put that in the draft. Or you can rely on meeting points or 07:37.600 --> 07:44.680 poor request reviews in order to make sure that a change to draft is approved by 07:44.680 --> 07:51.320 a majority of the group or by any kind of means that you would like to set up. 07:51.320 --> 07:54.640 And yeah as I was saying you can absolutely add form a review process before 07:54.640 --> 08:01.600 publication etc. And so why does that work? Well it makes it just 08:01.600 --> 08:11.680 easy for everyone to participate. As an example during the early phase of the 08:11.680 --> 08:21.760 work project we were working on a proposed spec and I had concerns about 08:21.760 --> 08:28.720 one part of the spec that felt like it was going a little bit too broadly in one 08:28.720 --> 08:35.840 area and I bumped into the main experts of that particular aspect at a 08:35.840 --> 08:40.480 conference and literally tagged them on the draft the issue and they had to look 08:40.480 --> 08:43.560 the next day and they told me oh yeah that wouldn't work for us they would 08:43.560 --> 08:48.800 create this in this issue. So and essentially less than 24 hours I could get 08:48.800 --> 08:54.880 effective useful feedback directly in the document itself and be able to 08:54.880 --> 09:00.400 iterate over it. And it allows earlier involvement right you can get key stakeholders 09:00.400 --> 09:05.800 in the process at a very early stage and provide use cases and tell you what 09:05.800 --> 09:11.000 works and what doesn't what already exist in that feeling what doesn't. As I 09:11.000 --> 09:15.040 was saying it in the bills easier involvement of the main experts and then 09:15.040 --> 09:20.000 lastly I mean you're getting much much better tooling to manage common 09:20.000 --> 09:26.880 synchronism. I mean I I'm still on a bunch of groups that talk about 09:26.880 --> 09:32.200 since I'm like an Etsy and I kind of feel like half of me chuckles to be 09:32.200 --> 09:37.120 fair and the other half of me really feels sincere pain for the poor 09:37.120 --> 09:40.600 repertoire is who are dealing with thousands of comments that they have to 09:40.600 --> 09:44.640 integrate and figure out like whether they conflict or not because the tooling is 09:44.640 --> 09:50.840 so bad for us right if you're building on top of tooling that's been used and 09:50.840 --> 09:56.680 we'll find for decades to manage code mergers and it is really easy to do 09:56.680 --> 10:03.920 those. And I think that's all I have and I can now take a bunch of questions and 10:03.920 --> 10:10.480 get thrown tomatoes at. Thank you very much for having me. 10:14.640 --> 10:26.520 Hi, so thank you very much for this explanation. I would have left for 10:26.520 --> 10:32.040 for example IETF's and the station to be the way you describe it where people 10:32.040 --> 10:36.800 are eagerly do in fighting on how to manage the documents so comment 10:36.800 --> 10:41.360 resolution and IETF is also sometimes not fun on the other hand for example 10:41.360 --> 10:45.520 looking at the Etsy where some of the standards for U.S.R. are managing the 10:45.520 --> 10:50.040 public GitLab instance and you can set pull requests and they are written 10:50.040 --> 10:55.400 mark down. So I think the reality check is you'll would love it ideal word 10:55.400 --> 11:01.600 however there's no single standards arc which even has that so do you 11:01.600 --> 11:06.480 recommend more for example IETF to move more into the exit direction of 11:06.480 --> 11:10.360 having everything in the GitLab or is that not intentional. 11:10.360 --> 11:16.480 So I actually I'm not super familiar with how ITF works because I'm mostly 11:16.480 --> 11:22.720 I've been more involved with W3C and ECMA than ITF and my understanding is 11:22.720 --> 11:27.640 ITF is very flexible in terms of tooling and different communities they 11:27.640 --> 11:35.800 are used different things but I must say I can't really talk for them. 11:35.800 --> 11:40.120 I'm not sure I the sound is terrible I don't know if you can actually if you're 11:40.120 --> 11:43.440 on the sort anything I said but like the questions are just really a 11:43.440 --> 11:46.640 point to even understand from here so I'm not sure I understood your question 11:46.640 --> 11:48.800 well and answered it properly. 11:48.800 --> 11:56.160 Yeah so I think one of the things which you made a very good point of is that 11:56.160 --> 12:00.760 we need batch of preparation tools and pretty much every standards process 12:00.760 --> 12:06.240 breaks down and I think you outlined some very good ideas we just need to make 12:06.240 --> 12:10.480 sure they are applied everywhere and I think that's the benefit of your great 12:10.480 --> 12:15.280 benefit of your talk you've point out how to improve tooling in a way which makes 12:15.280 --> 12:17.520 people not unhappy but productive. 12:17.520 --> 12:19.360 Yeah thank you very much for that. 12:19.360 --> 12:23.600 Thank you. 12:23.600 --> 12:30.640 I think I'll get you all to you because I know all to me Simon wants to exercise. 12:30.640 --> 12:36.960 I have to get what's that count up. 12:36.960 --> 12:39.160 Thank you very much. 12:39.160 --> 12:42.640 Ultramodic and from it yeah thank you. 12:43.600 --> 12:51.440 Actually it's very interesting question. 12:51.440 --> 12:56.440 It was on and it stopped okay yes ultimately again from it's very interesting 12:56.440 --> 12:59.600 what you're presenting thank you very much I agree with about 90% of what you've 12:59.600 --> 13:04.760 said and it's true that from the very beginning when I started looking in detail 13:04.760 --> 13:08.880 an open source I realized that actually the open source community their ways of 13:08.880 --> 13:12.520 working the spirit behind it was not actually very different from the world 13:12.560 --> 13:17.720 of standards and the tooling is different that's for sure but the idea that 13:17.720 --> 13:22.520 everybody works together to build something that is collectively better by 13:22.520 --> 13:26.840 working together and delivering it publicly that's more or less the same spirit 13:26.840 --> 13:30.880 the spirit of collaboration it's there there is one thing I would have an 13:30.880 --> 13:34.600 issue with and that's you statement at the beginning of standards are slow that 13:34.600 --> 13:40.400 depends on where you look one of the biggest delays in writing standards 13:40.400 --> 13:45.600 and has been the case with the CRA standards has been the time it takes to get 13:45.600 --> 13:51.320 consensus the time it takes to get input and to get that input agreed in the 13:51.320 --> 13:54.320 committee that's what takes time the actual editing and what not that's 13:54.320 --> 13:59.240 mechanical yes it's expensive it can be time consuming and annoying but that's 13:59.240 --> 14:04.600 not where the delay is the delay isn't getting consensus we among other 14:04.600 --> 14:09.720 things develop the standards for the mobile communications the 3G 4G 5G and 14:09.720 --> 14:13.360 soon 6G we do it collaboratively with other standards bodies in the world 14:13.360 --> 14:17.840 at the moment we're assembling all of the technologies that will go into 14:17.840 --> 14:23.640 6G in what we call study items in a third generation partnership project by 14:23.640 --> 14:29.480 in mid twenty twenty seven we will start writing the standards for 6G in about 14:29.480 --> 14:33.800 two two and a half years we were right about fifteen hundred standards some of 14:33.800 --> 14:38.600 them up to a thousand pages long describing the entire 6G system we'd have 14:38.600 --> 14:42.120 about seven or eight hundred companies organizations involved in that 14:42.120 --> 14:47.720 probably about two or three thousand experts sometimes a thousand people in 14:47.720 --> 14:53.280 the room going through a thousand two thousand documents per week doing all 14:53.280 --> 14:57.400 of that online we've tried it very difficult to get consensus across 14:57.400 --> 15:01.160 a thousand people in an online session and in the time frame that we need to 15:01.160 --> 15:06.120 move so you know we're going to move very fast with this when those standards 15:06.120 --> 15:10.200 are delivered they're updated four times a year with again thousands of 15:10.200 --> 15:14.440 changes tracking what actually happens in the mobile systems so there are 15:14.440 --> 15:18.640 parts of standardization that are very slow that's true there are parts that 15:18.640 --> 15:22.960 can move much faster than anything we've we're used to seeing even in the 15:22.960 --> 15:27.320 open source work yeah no I mean I you know I entirely I entirely here to 15:27.320 --> 15:32.360 point and I mean just to be clear like I'm literally quoting from a report 15:32.360 --> 15:36.760 that is the commission's work it like I have you know and based on like 15:36.760 --> 15:43.920 so I'm not making a statement here of my perspective on it and they're I 15:43.920 --> 15:50.840 also don't want to suggest that there was a one-size-fits-all solution but 15:50.840 --> 15:55.720 clearly my experience of at least a synthetic work and I'm around the 15:55.720 --> 16:02.400 series I'm very honestly not familiar was how the Etsy one went because I 16:02.400 --> 16:08.800 was not in the rooms but that was like really not great to put it mildly 16:08.800 --> 16:20.960 and and I think the process used there is not adapted to the kind of standard 16:20.960 --> 16:26.760 that it's supposed to be I don't know how to tell you know the telcomes industry 16:26.760 --> 16:30.320 works and I don't know how they build standards and it's also very different 16:30.320 --> 16:36.200 because it's a structure that as far as I know there's a lot of IP involved in 16:36.200 --> 16:40.840 the standard processes and the structure is it's very different so it's totally 16:40.840 --> 16:46.280 normal that four some solutions some systems work and for others others are 16:46.280 --> 16:53.640 better and I think it's good when we can use the right tools for the job so yeah 16:53.640 --> 17:01.880 thank you for your content I'm kind of thinking about how this oh sorry I'm 17:01.880 --> 17:10.960 Daniel Aaronberg working as a vertical rapture for browsers in Etsy and I'm 17:10.960 --> 17:18.800 wondering about how this recommendation about tooling and processes seem I'm 17:18.800 --> 17:25.840 wondering about how we can the the European structures we we have these various 17:25.840 --> 17:34.840 kind of comment periods as large it's been to early in the there's kind of this 17:34.840 --> 17:38.920 initial open period and then there's this these other official like public 17:38.920 --> 17:43.960 degree periods I think it it could be interesting to figure out how we could 17:43.960 --> 17:49.000 map those to the the kind of open source process as you continuously have comments 17:49.000 --> 17:59.960 coming in and you're continuously resolving them which kind of it's the you know 17:59.960 --> 18:05.840 seek large comment review processes I'm not sure how we could adapt one to the 18:05.840 --> 18:19.720 other but they're they're like trying to estimate so yeah sure should I go 18:19.720 --> 18:26.240 to yeah okay so I don't know the specifics of how the process of at sea 18:26.240 --> 18:33.160 works and when there's you know specific review a period etc the whole idea of the 18:33.160 --> 18:39.240 way standards that are built on using an open source source collaboration model 18:39.240 --> 18:46.360 are done is that you can still add like formal review periods when you want to 18:46.360 --> 18:50.640 at the end of the process right but you start with the review process from the 18:50.640 --> 18:59.200 get-go because everything is in the open and the benefit of that is you essentially 18:59.200 --> 19:04.160 like you know the the whole comments they come like over a much longer period of 19:04.160 --> 19:11.560 time and you can tune the document as you're getting input rather than do a 19:11.560 --> 19:17.000 draft sort of kind of by yourself because I mean you know the reality of how 19:17.000 --> 19:20.320 these standards are of written it's kind of like there's a reporter and like 19:20.320 --> 19:25.360 they do most of the work right and and then once you've done most of the 19:25.360 --> 19:28.840 work then get yelled at by lots of different people that are unhappy was it 19:28.840 --> 19:33.760 and have to sort of like scramble to get that together right so essentially you 19:33.760 --> 19:39.400 get yelled at less but for much longer period of time I don't know if the IPRs 19:39.400 --> 19:43.680 you know the the process structure of at sea would allow for this but if it 19:43.680 --> 19:51.880 did it would be great I told you hello presentation so I will be very brief 19:51.880 --> 19:58.120 I'm aware of time so I'm Lucie Landrie yes we can introduce myself from 19:58.120 --> 20:04.200 Cincinnati actually I took a picture from a presentation it was something I 20:04.200 --> 20:09.120 loved rules should be designed to ensure force rule roll as a public good 20:09.120 --> 20:14.920 as a central infrastructure and I think in the standards culture we are 20:14.920 --> 20:19.920 a little bit far from there from being there from understanding the importance of 20:19.920 --> 20:24.480 force and I loved all of your ideas for how to work together but so you 20:24.480 --> 20:31.560 think also the lack of awareness sometimes or maybe not yeah that also takes 20:31.560 --> 20:38.880 a place a role in a contribution start taking into account I'm sorry I missed 20:38.880 --> 20:44.640 the last bit of that and it's not nice if you know if you thought maybe it 20:44.640 --> 20:49.080 was also possible that the lack of awareness from stakeholders may 20:49.080 --> 20:52.720 contributions difficult to be taken to account doing working-grabbed 20:52.720 --> 21:01.640 discussions I'm not sure I understood the I'm so sorry Lucie awareness of how 21:01.640 --> 21:12.840 important force is for as a critical infrastructure for all yes yes yeah so 21:12.840 --> 21:15.120 this is this is yes so now I'm much more than the same 21:15.120 --> 21:19.800 comment yes and this is precisely why if you do things in the open you kind of 21:19.800 --> 21:25.080 like get this input and this feedback earlier on and it's kind of more obvious 21:25.080 --> 21:29.880 right like if you have lots of people that are bringing sort of like technical 21:29.880 --> 21:37.880 arguments to the table to you know for one way or another it makes it a lot easier 21:37.880 --> 21:46.120 to you know like see them and then address them and so in closing thank you 21:46.120 --> 21:53.080 all for your comments very much so I really hope that we're able to get 21:53.080 --> 21:59.040 processes that work better for everyone and hopefully you didn't take those 21:59.040 --> 22:07.440 comments to badly and I'm more than happy to work with you all to you know 22:07.440 --> 22:12.520 help move this forward thank you very much folks