[Development] Dev needs to be optimised

[Pollux]

I don’t think shell access is really needed plus just because you can script Lua doesn’t mean you know your way around a shell, especially since it’s all managed via docker, could FTP servers be setup inside the docker containers instead for security? That way devs don’t get access to the dedi’s full filesystem, just the gmod server directory

Pterodactyl already puts the entire server into a docker container and provides a per-user FTP account which you can use to access files within that container. We were refused it anyway as it isn't logged and we could "abuse" it. I've already spent a lot of my time here on FL arguing about dev changes and pushing them back with dev strikes, it's a constant battle that I kinda gave up with a while ago as I've been focusing on other projects since which I have much more control over and I have also had uni coursework I've been focusing on.

I stopped caring about it a while ago but it is ironic how the only developer on the team who has ever abused their trust massively has full access and is in control of everyone else.

[TASSIA]

I don’t think shell access is really needed plus just because you can script Lua doesn’t mean you know your way around a shell, especially since it’s all managed via docker, could FTP servers be setup inside the docker containers instead for security? That way devs don’t get access to the dedi’s full filesystem, just the gmod server directory

Looking at the sheer amount of times features of the "correct git flow" (for example LFS) simply fail to work just proves that sometimes you need to log in with a shell and pull changes forcefully. It is also ironic how for years developers, who invest their freetime into trying to make FL better, always have to fight against distrust, while apparently developers who actually abused their powers to great extent still get a 2nd chance in one of the highest positions. It just doesn't quite make sense to me.

Also, I can vouch for at least 3 current developers (and even more if we included retired ones) that are well able to use shells. If you genuinely only know Lua and have never touched another programming language and system infrastructure before, I'm not sure one is even fitted for the developer rank. Yes, in enterprise-level companies you require developers who only know one specific area, and thus it is fine there. But as mentioned previously, FL is so far from enterprise, and developers literally need to be experienced in so many areas. I remember when FLs harddrive crashed and Conn (I think it was him) had to literally spent days trying everything to restore any data, and actually succeeded.

[Hooter]

Perhaps a percentage of each donation goes to a hypothetical "dev fund"?

[Pollux]

Perhaps a percentage of each donation goes to a hypothetical "dev fund"?

30% is meant to.

[User 7141]

I don’t think shell access is really needed plus just because you can script Lua doesn’t mean you know your way around a shell, especially since it’s all managed via docker, could FTP servers be setup inside the docker containers instead for security? That way devs don’t get access to the dedi’s full filesystem, just the gmod server directory

I stopped caring about it a while ago but it is ironic how the only developer on the team who has ever abused their trust massively has full access and is in control of everyone else.

Yikes

There’s a lot coming out in this thread and I don’t understand why any of you put up with it frankly

[De CodeerHeer]

Completely unrelated but which certs did you do? I'd kinda like to have git certified on my list, even just as a training thing, even though I hate git with every inch of my body

The company I work at supplied all these courses and certificates. We are required to have full knowledge of the tools we work with in order to operate smoothly within any of the development processes.

I strongly disagree with your strong disagreement.

I think everyone who has worked with the gamemode before knows how unstable it is and that it has a tendency to crash frequently. This alone requires devs to at least kill and restart servers, in case the so-called "DevOps developers" are unavailable, e.g. because it's 4 AM.

But that's not even the main reason I disagree. Developers (in case of them being active, so excluding you) spent lots of their freetime working for this community and receive basically no payment. So it's even more important to keep developers motivated. Restricting them in everything hurts this in my opinion. I remember the struggle of being unable to fix various bugs on the website because I simply had no access to it, and the developers having access were inactive.

As said we have a separate server panel that is accessible and managable to a certain extend. The limit being access to the point where things can get even messier than they currently are and can cause confusion to infrastructure of the system.

I'm sorry you have had a bad experience with inactive developers being in "charge" of a certain department. But you have to understand that everyone has its expertise and experience with their subject. Whilst I praise the possibilty to widen and extend your coding capabilities and knowledge, it still has to be safe at the end of the day to secure the data. Which is one of the points for setting up a good workflow with reviews. Where such things can be resolved before reaching production. This is a much safer process of development and my reasoning for implementing it. This has nothing to do with enterprise standards. This is care a developer should have for it's work. Sure it might not be the quickest solution. But its the safest, and that is necessary for a community with a size like this.

Looking at the sheer amount of times features of the "correct git flow" (for example LFS) simply fail to work just proves that sometimes you need to log in with a shell and pull changes forcefully. It is also ironic how for years developers, who invest their freetime into trying to make FL better, always have to fight against distrust, while apparently developers who actually abused their powers to great extent still get a 2nd chance in one of the highest positions. It just doesn't quite make sense to me.

The initial problems with git lfs was it being setup by someone who was new to LFS and didn't have the required experience to set this up, being one of the reasons why server maintenance has been assigned to DevOps.

2nd chances exist to show that was you did was wrong and regret your actions. But still have the interest of helping my community with the skills and knowledge you possess. Same for me. It's not like I didn't had to earn this trust again.

I'm also not sure about your "ticketing system". Yes, it's important to properly test and get approvals for big updates, however, I think that is really unnecessary for minor updates or bug fixes and only hurts the development workflow and motivation of developers. Something I can see you benefit from, is having a GitHub action that checks the code for syntax errors and blocks PRs from being merged if they contain any. But that is not a thing to this day, afaik.

You need to understand that some actions have been taken due to feedback from the community. The sole purpose of contributing is to take approved suggestions and implement them into the gamemode. When the community complains you have to take action and revert to how the system supposed to work. This is done via tickets so you have control on what goes in. I find it facinating how the owner and management barely has a say on what is being implemented just because there is no grip on development. This is not how it is supposed to work. You don't add features the client hasn't asked for?

Back in my day I had to ask permission for everything. Implementing new features were done via a teamviewer session on the owners pc. These days we have a lot more access, but still some things are not for us to decide.

While in enterprise it is important to have clear roles, I don't know anything that could be further from enterprise than FL. I think the way you are doing actually slowly kills active development.

Even outside enterprise it is important to respect each others expertises and knowledge. If that means it will be safer, its a far better solution. If you don't think thats top priority, I doubt your intensions.

Also, I can vouch for at least 3 current developers (and even more if we included retired ones) that are well able to use shells. If you genuinely only know Lua and have never touched another programming language and system infrastructure before, I'm not sure one is even fitted for the developer rank. Yes, in enterprise-level companies you require developers who only know one specific area, and thus it is fine there. But as mentioned previously, FL is so far from enterprise, and developers literally need to be experienced in so many areas. I remember when FLs harddrive crashed and Conn (I think it was him) had to literally spent days trying everything to restore any data, and actually succeeded.

I'm not saying that there is a lack of shell knowledge. I'm saying that there lies a lot more behind the shell that you might not have enough knowledge of. Setting up and managing a linux server goes far beyond basic shell operations. Mistakes made in this area can cause huge implications (data leaks or failure). When you have people on the team who actually have the knowledge and experience (cause it's their daily job) you'd have to respect that, especially when it's of high importance to keep it secure.

I wad involved with the data recovery process. Most of it went lost or deemed unusable. The actual data recovery was unsuccessful. Almost everything was restored from off-site  backups or was reprogrammed.

Everything has it's reasons, being fair or not. We still have a community to serve at the end of the day, and we all have the intentention to this as good as possible. In order to keep that going well, we all have our own expertises and roles within the team. A role we should stick to. Just because you're an admin, and you can code a little doesn't make you a programmer. But that doesn't mean you can't become one. The most beautiful thing about a community like this is the ability to contribute freely. Learn from each other, extend your horizon. But respect the knowledge one another has. Understand that some can have more knowledge about a certain subject, and it might be better to leave them to the more dangerious tasks.

I contribute with this belief. I want to create a safe environment where we can connect and collaborate. Respect each other, and learn from them. Where you are free to express your expertise and share your knowledge. That is the community I found 9 years ago, and that is the community I keep fighting for.

[TASSIA]

As said we have a separate server panel that is accessible and managable to a certain extend. The limit being access to the point where things can get even messier than they currently are and can cause confusion to infrastructure of the system.

I'm sorry you have had a bad experience with inactive developers being in "charge" of a certain department. But you have to understand that everyone has its expertise and experience with their subject. Whilst I praise the possibilty to widen and extend your coding capabilities and knowledge, it still has to be safe at the end of the day to secure the data. Which is one of the points for setting up a good workflow with reviews. Where such things can be resolved before reaching production. This is a much safer process of development and my reasoning for implementing it. This has nothing to do with enterprise standards. This is care a developer should have for it's work. Sure it might not be the quickest solution. But its the safest, and that is necessary for a community with a size like this.

I totally see your point and I mostly agree. In modern software development all projects should be having quality assurance via approvals, test suites, GitHub actions analyzing & linting the code, and more. However, I just don't think that FL really could benefit from this. As said, a simple linter that checks for syntax errors is fine. Anything beyond that, not really. You are messing with a gamemode that is more than 10 years, based on a special Lua variant, has been contributed by a dozen of developers, each using their own code style for a game whose internal libraries have been updated massively. The only way to really get a safe, clean and good code would be to literally make a new gamemode. Being against such strict enforcement has nothing to do with being careless. If I wouldn't care I would just bypass the system somehow or just stop developing. I just do think that this really stops developers from creating updates quickly. If you really want to check every PR, you can still disable your checks and check the code afterwards and then revert the merge if you find something odd. Or you can already take a look at the branchs commits while the update is still a work in progress.

I just know that if I'm staying up until 3 AM to finish an update for FL, I want to merge it. And I want to merge it as soon as I fixed the last bug. I do not want to wait for approvals for the code that I have been testing and debugging the past 16 hours.

As far as I'm aware you have reduced the amount of required reviews from 2 to 1, which tbf, I quite like. I think with one required review even I could agree with, but 2 was just completely overkill. Still I would an option, if not already in existence, to instantly merge a PR if it's marked as a hotfix. As said, one can always check on it the next morning.

The initial problems with git lfs was it being setup by someone who was new to LFS and didn't have the required experience to set this up, being one of the reasons why server maintenance has been assigned to DevOps.

As far as I know, the person who was setting it up was in the middle of improving it, when their SSH access got taken away and a new system was promised. This new system either hasn't arrived, or, as could be seen today, is also non-functional (or well, at least buggy).

You need to understand that some actions have been taken due to feedback from the community. The sole purpose of contributing is to take approved suggestions and implement them into the gamemode. When the community complains you have to take action and revert to how the system supposed to work. This is done via tickets so you have control on what goes in. I find it facinating how the owner and management barely has a say on what is being implemented just because there is no grip on development. This is not how it is supposed to work. You don't add features the client hasn't asked for?

Back in my day I had to ask permission for everything. Implementing new features were done via a teamviewer session on the owners pc. These days we have a lot more access, but still some things are not for us to decide.

I find it fascinating how apparently now the issue is the owner and "management" having no say on what is being implemented, while back in 2019 we had to literally beg "management" and admins for ANY form of feedback, usually with far less than 50% even just responding. Furthermore, I think strictly binding a developer to a single project/feature kills development too. If I'm being forced to work on a feature requested by the "management", I'd probably have close to 0 motivation. If, for example, I could properly talk with "management" about what they think is needed currently and about what I currently would like to work on, one could find a way better consensus, and it would allow developers to still choose the project they would like to work at. Most of the time, the projects developers want to work on actually are approved suggestions. But in the rare occasion of a developer bringing up a completely new idea, it usually also turns out quite well.

Also, I would never address as FL as "my client".

2nd chances exist to show that was you did was wrong and regret your actions. But still have the interest of helping my community with the skills and knowledge you possess. Same for me. It's not like I didn't had to earn this trust again.

Even outside enterprise it is important to respect each others expertises and knowledge. If that means it will be safer, its a far better solution. If you don't think thats top priority, I doubt your intensions.

My intentions are good. I've spent an average 6 hours a day for 2 months working on a brand new automated documentation system for the gamemode, so new contributor would find their way around easier. After getting essentially blanked by the "web developers" I literally just gave up. This was probably one of the most disappointing points during my time at FL. I was simply not trustworthy enough to receive web access (tbf: I received FTP web access a few months later, but was fed up then). And this has nothing to do with me not respecting another ones expertise or knowledge. Instead, I respected the expertise and knowledge of the current web developers (which actually was only moderately good).

If you genuinely prefer this "safer" solution, fair enough. But understand that I does not help in motivating free time developers. You should at least give developers that also are experienced in a specific area to branch out into that area.

I'm not saying that there is a lack of shell knowledge. I'm saying that there lies a lot more behind the shell that you might not have enough knowledge of. Setting up and managing a linux server goes far beyond basic shell operations. Mistakes made in this area can cause huge implications (data leaks or failure). When you have people on the team who actually have the knowledge and experience (cause it's their daily job) you'd have to respect that, especially when it's of high importance to keep it secure.

True.

I wad involved with the data recovery process. Most of it went lost or deemed unusable. The actual data recovery was unsuccessful. Almost everything was restored from off-site  backups or was reprogrammed.

Considering the forum and ingame data was essentially completely restored, I'd still call it a successful recovery, regardless whether the majority was restored off-site or reprogrammed.

Everything has it's reasons, being fair or not. We still have a community to serve at the end of the day, and we all have the intentention to this as good as possible.

I totally agree with you. I'm just not sure the way it is currently handled gives the best result.

In order to keep that going well, we all have our own expertises and roles within the team. A role we should stick to. Just because you're an admin, and you can code a little doesn't make you a programmer. But that doesn't mean you can't become one. The most beautiful thing about a community like this is the ability to contribute freely. Learn from each other, extend your horizon. But respect the knowledge one another has. Understand that some can have more knowledge about a certain subject, and it might be better to leave them to the more dangerious tasks.

This sounds like a great vision. Sadly, when I was at FL, this vision was far from becoming reality. Not sure how easy it is to actually get involved in a different area nowadays, but I've heard that admin-developer communication has increased a bit.

I contribute with this belief. I want to create a safe environment where we can connect and collaborate. Respect each other, and learn from them. Where you are free to express your expertise and share your knowledge. That is the community I found 9 years ago, and that is the community I keep fighting for.

Consider running for presidency.

Jokes aside, I like your vision. But literally taking developers accesses away, at least to me, feels more like disrespecting them, if anything.

[User 7141]

I'm not saying that there is a lack of shell knowledge. I'm saying that there lies a lot more behind the shell that you might not have enough knowledge of. Setting up and managing a linux server goes far beyond basic shell operations. Mistakes made in this area can cause huge implications (data leaks or failure). When you have people on the team who actually have the knowledge and experience (cause it's their daily job) you'd have to respect that, especially when it's of high importance to keep it secure.

But even with these systems in place, there was a massive data leak the other week which just proves, with the scale of the leak that the systems currently in place don't properly work the way they've been designed to. If developers are trusted with the main product of FL (The gamemode) or in some cases the website I just can't put my finger on why there isn't some sort of better system for them to test code on, even if it was just a script that detected pushes to the git and pulled them down into the test server.

While it hasn't directly been said it's obvious that some or even all of the developers are getting demotivated and tired of having to fight to just get something they've written, for free, any sort of approval to be added into test/prod. With disgruntlements in this area it leads to people being sloppy amongst other things.

The only way I can see the motivation returning is if the SA/Management and dev team sit down together and discuss what they want from each other and I personally don't think this thread is the place for that to happen due to the sensitive nature of what might be discussed. It doesn't have to be full root access to the dedibox, but I think the dev team would appreciate being heard properly

[Conn]

Point A

Looking at the sheer amount of times features of the "correct git flow" (for example LFS) simply fail to work just proves that sometimes you need to log in with a shell and pull changes forcefully. It is also ironic how for years developers, who invest their freetime into trying to make FL better, always have to fight against distrust, while apparently developers who actually abused their powers to great extent still get a 2nd chance in one of the highest positions. It just doesn't quite make sense to me.

The initial problems with git lfs was it being setup by someone who was new to LFS and didn't have the required experience to set this up, being one of the reasons why server maintenance has been assigned to DevOps.

2nd chances exist to show that was you did was wrong and regret your actions. But still have the interest of helping my community with the skills and knowledge you possess. Same for me. It's not like I didn't had to earn this trust again.

Originally I began the integration of Git LFS in order to get a system set up quickly, as developers were stuck waiting around due to access to FTP being taken following a decision made by management & yourself. This decision was problematic due to there being no timely plan in place for dev access, which was necessary for multiple updates at the time. This lack of foresight has been shown multiple times, I will cover this further later.
LFS had some initial issues due to permissions, which I was working to fix, with the help of yourself. Admittedly, I was coming in with little experience with Git LFS, hence I was communicating with both you and Ed to ensure that stuff was done correctly when reworking to fix issues. I'd have happily stepped aside and allowed you to set this up, except time and time again it has shown that stuff isn't done in a timely manner, when it really needs to be.
The nature of the development on this community is sporadic, it happens when people get motivation to work on stuff, and if they're forced to climb over roadblocks the whole way there, they will not work on stuff.

My real problem comes with the absolutely embarrassing handling of this from yourself. This is best shown with the communications that we had, following you answering some questions that helped me sort out the issue:


Context: I was working on the revised script following the information given to me, and at the time, was around 6 hours in to doing so. Suddenly, the server randomly restarted, following by me losing access to my SSH twice. As it turns out, you removed my access with no right to do so.

The script that I was revising was originally written around a year ago, as a temporary solution to the Git auto-deployment system, which you assured was a top priority for you. As it turns out, that script had a potential vulnerability, which I accept, and was working to improve in the recent revision. Instead of speaking to me about said vulnerability, you spoke straight to management, working behind my back, at no point thinking it may be a good idea to speak to me, to work in a team, and to help resolve said issues. 

Fast forward to 27/03, where this is summed up quite nicely. 



You continue to show a refusal to work in a team, to work behind our backs for no good reason. I believe I speak for other developers when I say that the attitude that you have displayed towards the development team is nothing short of shocking. 

Point B

Also, I can vouch for at least 3 current developers (and even more if we included retired ones) that are well able to use shells. If you genuinely only know Lua and have never touched another programming language and system infrastructure before, I'm not sure one is even fitted for the developer rank. Yes, in enterprise-level companies you require developers who only know one specific area, and thus it is fine there. But as mentioned previously, FL is so far from enterprise, and developers literally need to be experienced in so many areas. I remember when FLs harddrive crashed and Conn (I think it was him) had to literally spent days trying everything to restore any data, and actually succeeded.

I'm not saying that there is a lack of shell knowledge. I'm saying that there lies a lot more behind the shell that you might not have enough knowledge of. Setting up and managing a linux server goes far beyond basic shell operations. Mistakes made in this area can cause huge implications (data leaks or failure). When you have people on the team who actually have the knowledge and experience (cause it's their daily job) you'd have to respect that, especially when it's of high importance to keep it secure.

We're in a small community full of volunteer developers. Mistakes can, and will be made. There will be people who are more experienced, and people who will be less experienced, but with working as a team, we can succeed it a much quicker and greater fashion than if there is a constant need for the best to be the only ones that have the capability to do something, especially when plagued with inactivity. 

I personally find it insulting to have been so blind sighted following a vulnerability in a year-old piece of code I wrote as a temporary fix, especially considering amateur blunders such as leaving indexing on, writing code with multiple XSS vulnerabilities, and so on. Mistakes will happen, we have to work as a team to fix them, and work as a team on the project as a whole. 

Point C

I wad involved with the data recovery process. Most of it went lost or deemed unusable. The actual data recovery was unsuccessful. Almost everything was restored from off-site  backups or was reprogrammed.

This is factually incorrect. Both gamemode database, and forums were recovered based on that data recovery effort. To put this into context for those who aren't aware: we lost around 2 months worth of data rather than 12 years worth - if this is not a success, I don't know what is.
The reprogramming of many features was a choice, rather than a necessity; though I don't think it was a bad choice in many cases. 

Point D

Everything has it's reasons, being fair or not. We still have a community to serve at the end of the day, and we all have the intentention to this as good as possible. In order to keep that going well, we all have our own expertises and roles within the team. A role we should stick to. Just because you're an admin, and you can code a little doesn't make you a programmer. But that doesn't mean you can't become one. The most beautiful thing about a community like this is the ability to contribute freely. Learn from each other, extend your horizon. But respect the knowledge one another has. Understand that some can have more knowledge about a certain subject, and it might be better to leave them to the more dangerious tasks.

I contribute with this belief. I want to create a safe environment where we can connect and collaborate. Respect each other, and learn from them. Where you are free to express your expertise and share your knowledge. That is the community I found 9 years ago, and that is the community I keep fighting for.

Point D funnels into my final point here. You cannot preach for such things without actually showing it yourself. As a team we worked extremely hard to recover from the data loss last year, and we have worked together to consistently improve the quality of development, however, with your step up, originally coming with a title of 'development co-ordinator', the development team have been constantly and needlessly blind sighted. 

It truly is unfortunate that it has to come to posting this on the public forum, but considering your unwillingness to tackle the issues in a conversation from March, I feel there's not much chance anything else will have any effect. There has been no respect shown from yourself when working behind our backs to remove access from developers. 
We need development team communication when working on workflow, when managing the infrastructure, and when working on projects. This won't work whilst there is such a divide in the team, that I personally believe has been created largely by yourself and your blind action.

[Pollux]

Considering what this has turned into, I'll move it into discussions.