🏻 👽 🏔️ FOSS News # 63 - a special issue about the "inner kitchen" of digests 👩‍👩‍👦 👩🏼‍💻 🥚

Hello everyone!

For more than a year now, I, together with several friends, have been making FOSS News digests for Habr with materials about free and open source software and hardware. Recently I was invited to an online conference for open source software developers "Admin", where I talked about our digests for those who have not seen them, and about their "inner workings" for those who are already reading. I thought that here the readers might also be interested in this, so I am publishing the text of the report on which I based my speech.

Today I want to talk about news digests and other materials about free and open source software and hardware called FOSS News. Free and Open Source Software News ie. My comrades and I have been making these digests since January 2020, that is, for over a year. In my speech, I want to not only tell what we are doing for those who have not read it, but also show some details of the preparation process, which will be new for those who already read us, and the rest may be interested. FOSS News is also not just digests about FOSS, it is a FOSS project in itself, because it has a lot of automation and its code is laid out under a free license. At PermLUG, I deal with organizational issues and projects, while my main job at Miro is a load testing engineer with about 12 years of experience in software development.

Background

It all started at the meeting of the Perm GNU / Linux Users Group on January 23, 2020, and the group itself appeared 24 years ago. I only went to school then and had not yet participated. Then there were 5-6 waves of activity and calm. I connected somewhere on the 4th. And the last wave was initiated by me, the idea of the digests was mine too, it was me in the top photo on the right with a strange face, there was simply no other photo. I wanted to do something useful besides meetings. I also began to do. Uniting is good, but uniting around a project is even better. And in record time, 3 days after that linuxing, issue # 0 was ready. And over time, a couple more people connected.

Idea

The idea behind the digests was simple. I read several sources about free and open source software and hardware, but I was not interested in the releases that received a lot of attention. I just regularly do sudo apt upgrade and am glad if something new appeared in the programs that I use, but if I haven’t noticed something, I’ll have to google it. It is the releases that I track literally for several projects. I wondered how the community still lives. What was implemented where, who opened the source, different analytics about FOSS achievements, fundraising, reports on the work of funds, and so on. The hypothesis was that I am not the only one interested in such prioritization in the first place.

Format

The format has changed over time and a little less than a year ago or so it became something like this:

Several main materials. It can be news, articles, or some particularly important releases.
Block "short line". Here, in the three indicated categories, everything that got into the feed from all sources and did not get into the main thing is collected, this is really a lot of materials, but there is just a heading and a link and everything is divided into categories. At the beginning of the digest there is a table of contents. You can just go to the desired block, skipping the uninteresting one.
Other. Here, for example, links to other digests or something else that did not fit into the main categories.

Released every week on Sunday evening.

Latest top stories

Here is an example of content from the last digest that I found important. As you can see, the topics are diverse and can be of interest to a much wider audience than the releases.

Development in a year +

In just over a year, 62 digests were released. Issue # 0 was published on VKontakte and it was a failure, there were very few views, only a few dozen, I was very upset. Now, remembering this, I think - why suddenly I decided to release VKontakte at all. Maybe just because we had a group only there and other thoughts could not come to mind.

Issue # 1 has already been posted on Habr on the advice of a friend. Thanks to the team lead from the last place of work for the invite. And here the work was rewarded - there were many views, 7 thousand. And a couple of hundred people immediately subscribed to the newly created channel in Telegram.

Further, we did not reach such a peak of views as in issue # 1 and, in general, the number of views stabilized, but a constant audience was formed and this is probably not bad either. The audience is about 2.5 thousand people - I don't know a lot or a little, I'm fine. Well, the Telegram channel is growing slowly but steadily, it's nice.

Preparation scheme

This is perhaps all that is prepared only for those who did not know about digests. More subscription links will be at the end. Now there is a little internal kitchen, it is also interesting, it has developed over more than a year of issues.

How digests are made:

An FNGS robot written in Python runs on the server and goes through a bunch of RSS and HTML sources, collects materials and filters by keywords.
- fncategorizer.py, Python ( Python), , , , , .
remotedatatohtml.py HTML.
HTML Google Drive, .
googledoctohtml.py , . , HTML.
, Telegram, RSS .
, .

And as I said at the beginning, the code of automation tools, both server and client, and auxiliary utilities is under a free license, links at the end. In principle, after some modifications, this can be used for any digests. True, I did not bother with this, but who needs to be helped.

Sources, keywords, filtering and more

A little more statistics on the slide. I collected Russian-language sources from memory, and I counted English-language sources a little, just some time ago linux.com collected thematic materials from other resources and made links on my site, and I just looked where they were collected from and added to the database. The keywords were typed while working with the categorizer that I wrote, and I also stole the names of all the projects that the Linux Foundation conducts. There are more sources in mind. Here it is written about 500 initial and 160 final materials - it's just that not all feeds are strictly thematic, you have to select by keywords, a robot does a lot of work here for me, I filter out literally a dozen or two manually or so. A dozen or two more are duplicates, then they have to be manually combined for now. By categories, the spreading is carried out in semi-automatic mode,the script makes me suggestions based on keywords, I accept or indicate my options.

For those who are interested - a complete list of sources (sorting just alphabetically):

Interactive categorizer

What a Linuxoid without a console :) Here is a screen of the categorizer client, with which I work every weekday in the mornings for half an hour or an hour. This screenshot may look scary for someone, but it's generally convenient for me, I'm glad when I use it, I greatly simplified my work.

Admin panel

Since the FNGS server runs on the Python Web framework Django, the admin area was a bonus. Through it, it is more convenient to make a search and edit and add some individual records. There is no particular merit here, I just show it. This is a team hodgepodge of console and graphical tools, but I'm fine.

Command

The FOSS News team consists of permanent members of the robot, together with Vasya from PermLUG and one more person from Habr. The robot does a lot of routine work, which if I had to do every week, I would definitely go crazy :) And I have never seen the last one in my life, it was just that he first threw off comments to the issues on Habré in a personal, and then I invited him to pre-release training, and he is a great fellow.

Plans

A long time ago, when I came up with FOSS News, I wanted not only to collect news, but also to provide context, links to related materials, in general, not just to collect links, but also to parse them fully. Unfortunately, I do not have time to do it well. So on the next slide I will urge you to join our team. If it succeeds, there will be more time and resources for the further development of the project.

Readers also suggested that we make video digests, but it seems too difficult to me, but podcasts - perhaps, although I myself do not listen to them.

And somewhere in the distant bright future, I would like to make a professional media out of FOSS News, I even bought domains already.

Join us!

As I already said, there are many plans, I want to improve the quality, open new directions, but there are not enough resources. Therefore, the idea arose to make distributed work on the longest part of the preparation process - the categorization of sources. To do this, you can make a Telegram bot. It can also be used to add materials that are out of sight of the FNGS robot.

Therefore, we invite Python developers who would like to help in the development of the bot, and just activists who would like to work with the bot in the future to participate.

We are also open to other options for cooperation.

And we will gladly tell you about your FOSS projects, throw off links!

Contacts and links

On the last slide there were contacts and links, I will give them just in text for convenience.

PermLUG resources:

Website (static page, just for links to everything else)
Telegram - @permlug (channel) and @permlug_chat (chat)
In contact with
Fediverse
Twitter
Facebook

Instrument sources:

My personal contacts:

gim6626@gmail.com
@ gim6626 on Telegram

Thank you for attention! Read, comment, distribute FOSS News and join our team!

Links to materials on the "Admin":

Full program te-st.ru/events/adminka
Day 1 Video www.youtube.com/watch?v=dvsCYfeo_qc
Day 2 Video www.youtube.com/watch?v=gCukiw4DWuU

FOSS News # 63 - a special issue about the "inner kitchen" of digests

Table of contents