orbiter-mods.com - announcing development of open source mods repo

Status
Not open for further replies.

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,358
Reaction score
513
Points
153
Location
Vienna
If by "fetch" you mean pipe the data through your server for user convenience, then you should talk to the OF staff if they'd consider that ok. Also, you should clearly mark in the download that the file will be delivered from another site.
If you just link to the OF page where the mod can be downloaded, then there's not actually a problem. There's not really any reasonable objection anybody could make to that.

So there might also be a communications issue here: From what you wrote so far, I always had the impression that you were planning to host the mods previously on OH, and I suspect most people thought the same. If you're not actively going to host the files, a lot of the issues go away. If you're not even piping the data through your server, there's no issues whatsoever.
I don't get the big issue here, either.
Did everybody discussing here actually try the site? It is a plain search form that lists result links in a plain manner, and if you click the link, you navigate to the O-F page for that add-on. It can't get anymore basic linking than that.
If the issue is that the site's cache is generated by fetching meta-data from O-F (in a resource-minimizing manner), then I really see some sort of monopoly thinking. Please don't say that we've staggered into that territory.
 

supersonic71

Member
Joined
Sep 20, 2021
Messages
59
Reaction score
83
Points
18
Location
Asia Pac
Website
github.com
In my opinion, it would be very very useful if orbiter-forum provides an api to download addons (directly to user's computer).
Single click installation of addon collection using manifest/recipe files would ease one of the biggest pain points of new users, which is manually and recursively downloading the dependencies one by one.
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,358
Reaction score
513
Points
153
Location
Vienna
In my opinion, it would be very very useful if orbiter-forum provides an api to download addons (directly to user's computer).
Single click installation of addon collection using manifest/recipe files would ease one of the biggest pain points of new users, which is manually and recursively downloading the dependencies one by one.
That would be an automated non-interactive usage of OHM/O-F data and is now discouraged by the new terms of service. In the discord chat, it was confirmed that there is no plan to have a public API that would allow for something like this.
 

jedidia

shoemaker without legs
Addon Developer
Joined
Mar 19, 2008
Messages
10,307
Reaction score
1,511
Points
203
Location
between the planets
Actually the implementation I was thinking of would not have any data going through my servers, and my servers would only host metadata. The idea was for the user's client machine to download everything, just in an automated fashion.
Then the whole thing seems mostly a misunderstanding to me. There should be no issues with that. It's possible the blacklist was put up in anticipation that you might try to crawl the space to download the zips. At least I hope so. I don't see any grounds for objection under these terms, unless there's expected technical issues with automated querying of metadata... and I don't think we can expect enough of a user base to make that a problem.
 

Majid

Active member
Joined
Oct 31, 2014
Messages
156
Reaction score
27
Points
43
I have absolutely NO PLANS whatsoever to crawl OF for the files.

In my opinion, it would be very very useful if orbiter-forum provides an api to download addons (directly to user's computer).
Single click installation of addon collection using manifest/recipe files would ease one of the biggest pain points of new users, which is manually and recursively downloading the dependencies one by one.

This is literally the goal of orbiter-mods.com, to be kind of the npm/AUR of orbiter. There are thousands of add-ons out there, most of which are obsolete. So short term plan is to provide scripted installs, long term plan is to create a dependency graph for purely scripted installs of addons and their dependencies.

That would be an automated non-interactive usage of OHM/O-F data and is now discouraged by the new terms of service. In the discord chat, it was confirmed that there is no plan to have a public API that would allow for something like this.

If OHM/OF has a problem with an orbiter addon fetching the files from OHM/OF on the target user system to auto install the addon then I think the staff's intentions and motivations are misguided.

BTW I don't want to distract from the main conversation but some info:
I fetched 26.9 MB (28,288,124 bytes) of data over a period of 5 mins. The index can be viewed here: http://orbiter-mods.com/addons.json
Didn't fetch any descriptions or anything else in order to minimize load.
 
Last edited:

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,358
Reaction score
513
Points
153
Location
Vienna
If OHM/OF has a problem with an orbiter addon fetching the files from OHM/OF on the target user system then I think the staff's intentions and motivations are misguided.
I'm a bit surprised as well, but the intention gets pretty clear with the ToS "Web-Browser-Only" statement.

My suggestion would be to not invest anymore time or energy in this project. It seems like your form of help is not appreciated.
 

Majid

Active member
Joined
Oct 31, 2014
Messages
156
Reaction score
27
Points
43
I'm a bit surprised as well, but the intention gets pretty clear with the ToS "Web-Browser-Only" statement.

My suggestion would be to not invest anymore time or energy in this project. It seems like your form of help is not appreciated.

I'll create an electron app and have users click somewhere. That'll check both the web browser + interactivity boxes ;)

I have to say the response I have received is very surprising. Mods uploaded to OHM belong to their authors, not OF. We desperately need a better user story for mod management because right now it's almost unusable.

So work on this will continue as planned, and if OF wants to penalize me so be it. I have been and am open to cooperation and working with people to build this into something valuable, and will continue to do so.

In case I get banned from here, discussions can transition to: https://github.com/computerex/orbiter-mods.com

I have a history of going AWOL, but I am here to stay and see this through. These conversations have only strengthened my resolve to build an alternative repo and improve user experience when it comes to orbiter mods.
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,358
Reaction score
513
Points
153
Location
Vienna
I have to say the response I have received is very surprising. Mods uploaded to OHM belong to their authors, not OF. We desperately need a better user story for mod management because right now it's almost unusable.

I agree. Your really simple search box alone shows how much better the UX could be.

So work on this will continue as planned, and if OF wants to penalize me so be it. I have been and am open to cooperation and working with people to build this into something valuable, and will continue to do so.

In case I get banned from here, discussions can transition to: https://github.com/computerex/orbiter-mods.com

I have a history of going AWOL, but I am here to stay and see this through. These conversations have only strengthened my resolve to build an alternative repo and improve user experience when it comes to orbiter mods.

Well, O-F staff can go cancel-crazy and ban you here, sure, but I doubt that. I also doubt that there will be any legal threats, at least not for what you did.
However, since they don't want your help (in this form), seemingly take measures against your site, and discourage probably helpful usage in the ToS, I fail to see how an outright "fight" between different repos will serve anyone in the community.

Just my 2 cents, of course.
 

Majid

Active member
Joined
Oct 31, 2014
Messages
156
Reaction score
27
Points
43
I agree. Your really simple search box alone shows how much better the UX could be.



Well, O-F staff can go cancel-crazy and ban you here, sure, but I doubt that. I also doubt that there will be any legal threats, at least not for what you did.
However, since they don't want your help (in this form), seemingly take measures against your site, and discourage probably helpful usage in the ToS, I fail to see how an outright "fight" between different repos will serve anyone in the community.

Just my 2 cents, of course.
Fighting is the last thing I want to do. I wanted to contribute through the “official” channel and messaged xyon. I would gladly work on the existing OHM site and make it better. I would gladly do any work on orbiter-mods under the umbrella of OF, all I ask is to let me make technical/ux related decisions.

I don’t want to divide, I realise I can be more productive by working together. I don’t know why I’m being perceived as a threat.
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,358
Reaction score
513
Points
153
Location
Vienna
I would gladly do any work on orbiter-mods under the umbrella of OF, all I ask is to let me make technical/ux related decisions.

Well, Xyon made her opinion pretty clear: she doesn't trust you in making proper decisions.

I don’t want to divide, I realise I can be more productive by working together. I don’t know why I’m being perceived as a threat.

If I read the post right, you are not really perceived as a threat, but simply not trusted because of your history. As it is, continuation can very well lead to conflict. It is a sad state of affairs, but it is what it is.
 

Majid

Active member
Joined
Oct 31, 2014
Messages
156
Reaction score
27
Points
43
Well, really any conflict that happened with me and the forum was I think in 2008-2010. I was like 15 years old. No one is more sorry than I for the way that I was when I was a kid. It literally happened over a decade ago. I haven't even used my real account, computerex, because I don't necessarily want to associate with that account. I am not a kid anymore.

At any rate, it is what it is, what'll happen will happen. I am trying to provision a wildcard cert with let's encrypt but haven't had the patience for DNS to propagate fully before continuing on with their verification. Always waited for "enough" servers to propgate and continued too soon. Have failed verification like 5 times now.

Will stop playing around and provision the cert soon, and will expect to put in some real hours for this over the weekend. Things to decide on is the runtime for the scripts that'll execute against the user's orbiter install. I am thinking Lua because orbiter might already come with the right lua executables.
 

jedidia

shoemaker without legs
Addon Developer
Joined
Mar 19, 2008
Messages
10,307
Reaction score
1,511
Points
203
Location
between the planets
I haven't even used my real account, computerex, because I don't necessarily want to associate with that account.
Wait, you're computerex? Wow, welcome back! I remember there having been some kind of ruckus, although I don't remember any specifics. What I do remember you as is "the crazy kid that pushed out one really good mod after another". Glad to see you back!
 

dbeachy1

O-F Administrator
Administrator
Orbiter Contributor
Addon Developer
Donator
Beta Tester
Joined
Jan 14, 2008
Messages
9,164
Reaction score
1,500
Points
203
Location
VA
Website
alteaaerospace.com
Preferred Pronouns
He/Him
The search was like half an hour worth of effort yall to allow users to get a decent search experience. There are no plans for this to stay, and in fact I can remove it right now if everyone wants.

The search is working again for those that want to use it.

FWIW, here is the search page for the new addon repository here on the forum:

Screenshot 2022-06-09 114308.png
 

Face

Well-known member
Orbiter Contributor
Addon Developer
Beta Tester
Joined
Mar 18, 2008
Messages
4,358
Reaction score
513
Points
153
Location
Vienna
FWIW, here is the search page for the new addon repository here on the forum:

I tried that with the term "OMP", and I got an "Oops, no results" response, presumably due to the term being too short. Majid's site did the search and returned proper results (after that initial bug was fixed). Overall, IMHO it has a much better "interface", resembling a kind of OHM-specific Google search with immediate response. It would really be cool to have something similar for the search here.
 

dbeachy1

O-F Administrator
Administrator
Orbiter Contributor
Addon Developer
Donator
Beta Tester
Joined
Jan 14, 2008
Messages
9,164
Reaction score
1,500
Points
203
Location
VA
Website
alteaaerospace.com
Preferred Pronouns
He/Him
I don’t want to divide, I realise I can be more productive by working together. I don’t know why I’m being perceived as a threat.

One of the reasons is that you dragged what was a private discussion started via @Xyon's direct message to you into your public thread here rather than discuss the matter further with the O-F Staff. Private messages, whether from staff or not, are just that, private, and posting them in a public thread without the consent of the other parties is not a good way to have honest discussions, because it demonstrates a lack of good faith. It is also a violation of O-F rules about discussing staff action in public, but that ship has sailed now.

Beyond that, the major issues that staff have is that you have stated publicly in this thread that your intention is (or was, now?) to create a repository where users could upload add-ons in addition to listing all the addons from the forum's resource section, and that is the source of the concern with the staff here for the reasons laid out earlier.

Here is a simple example of our concerns with that approach -- I am copying this from a post I made in the private O-F staff discussion thread about this yesterday:

So there's yet another key reason to prohibit allowing other repos to scrape the official O-F addon repository, and that is user verification and security. Here's a simple example of what could easily happen if we would allow other repos to scrape O-F. I'll use Computerex's site, Woo482's Themis add-on, and our legendary troll PAX as an example:
  1. Computerex's site continues to scrape the O-F Resources section.
  2. PAX or some other malicious user discovers Computerex's new repo. They download Woo482's Themis and deliberately infect the DLL(s) with malware.
  3. PAX then creates a new account on Computerex's repo named "Woo482". Since Computerex's repo does not have access to O-F authentication, that site has no way to verify that the "Woo482" who just registered is the real Woo482.
  4. PAX logs in to Computerex's repo as the fake Woo482 and uploads the malware-infected Themis, naming it "THEMIS 2.0".
  5. Hapless users come along and believe that Computerex's repo is the official repo, since it apparently has all the addons on it, INCLUDING what appears to be a new THEMIS 2.0 release from what appears to be Woo482 himself listed right next to the original Themis release.
  6. The users download and run the malware-infected "Themis 2.0" version.
  7. Chaos and recriminations ensue, and Woo482 has to clean up the mess.
That's just one simple example.

So to sum up, if your intention now is to only have a read-only site that indexes add-on metadata and links scraped from the Addon Repository here on Orbiter-Forum and does not allow uploads of new add-ons, then the staff have no issues with that. If, however, your site will allow uploading of add-ons in addition to showing add-ons scraped from the Addon Repository here, that would be a forum TOS violation.

If you would be willing to take these discussions private, which IMO would be more conducive to good-faith discussion, the O-F staff would welcome that. We have some other functionality that we would like to have, such as addon collections, that the forum's resources section currently does not support. This is where a separate, read-only indexing site could help, and it would complement the addon ecosystem rather than fragment it.
 

GLS

Well-known member
Orbiter Contributor
Addon Developer
Joined
Mar 22, 2008
Messages
5,010
Reaction score
1,644
Points
188
Website
github.com
I tried that with the term "OMP", and I got an "Oops, no results" response, presumably due to the term being too short.
Yes, the forum search doesn't work with 3 chars or less... which is a shame as it effectively "excludes" several spaceflight acronyms. :(
 

Majid

Active member
Joined
Oct 31, 2014
Messages
156
Reaction score
27
Points
43
So to sum up, if your intention now is to only have a read-only site that indexes add-on metadata and links scraped from the Addon Repository here on Orbiter-Forum and does not allow uploads of new add-ons, then the staff have no issues with that. If, however, your site will allow uploading of add-ons in addition to showing add-ons scraped from the Addon Repository here, that would be a forum TOS violation.

Are you saying that no one else besides OF can create an orbiter add-ons repo? Because that's what it sounds like. I am creating a completely independent user system, and only logged in users will be able to upload addons. Are you saying that when addon authors upload to OF/OHM, they are giving up rights to upload anywhere else on the internet?

Are you saying that if you are an OF user, you are giving up your rights to create a publicly hosted Orbiter mods repo?

My app will have uploads. Add-on devs will have the opportunity to share their work there, and every effort will be made to ensure that the uploads were done in good faith by the author or with the author's permission.

BTW, this is the reason why even OF needs a proper claim system: https://www.orbiter-forum.com/resources/imfd57-zip-for-anyone-that-can-39-t-find-it-anywhere.1562/

I am not sure if Jarmonik consents to this upload. What is OF doing to protect the rights here?
 

Majid

Active member
Joined
Oct 31, 2014
Messages
156
Reaction score
27
Points
43
Look yall, please fix the resource manager search, and I'll remove the search from my site. I have no intentions of starting a fight over this. I just want users to be able to search with things like "xr2". I have no intention of scraping anything from OF.

Also in response to the scenario presented by Doug: I wouldn't allow add-on uploads from totally new accounts that haven't been vetted. Any user name that matches an OF user would be properly vetted, probably by using OF as source of identity. I have posted this already somewhere in this giant thread :)
 

dbeachy1

O-F Administrator
Administrator
Orbiter Contributor
Addon Developer
Donator
Beta Tester
Joined
Jan 14, 2008
Messages
9,164
Reaction score
1,500
Points
203
Location
VA
Website
alteaaerospace.com
Preferred Pronouns
He/Him
Add-on authors are of course free to upload their add-ons anywhere they wish. The issue here is whether a given repository (i.e., the site to which those addons are uploaded) also lists those newly uploaded addons alongside the add-on entries it scraped from the Addons Repository on the forum here, for the obvious security reasons just detailed in the previous post. By listing newly uploaded addons alongside addons entries scraped from Orbiter-Forum, it's trivially easy for malicious users to impersonate real O-F users and upload fake new versions of those linked add-ons, and that will cause problems for the real add-on authors who originally uploaded their work to OHM.

To put it another way, it is of course not against forum TOS to create a completely separate site that accepts uploads, because the forum is not involved in any way. However, that same repository cannot scrape addon metadata and list those results in the same searches alongside the addons its hosts itself, because then the forum is involved: it's a huge security issue because then it becomes trivially easy for malicious users to impersonate the real add-on authors scraped from the Addon Resources section here and have those malicious links listed right alongside the real add-ons scraped from the forum.
 

dbeachy1

O-F Administrator
Administrator
Orbiter Contributor
Addon Developer
Donator
Beta Tester
Joined
Jan 14, 2008
Messages
9,164
Reaction score
1,500
Points
203
Location
VA
Website
alteaaerospace.com
Preferred Pronouns
He/Him
Any user name that matches an OF user would be properly vetted, probably by using OF as source of identity. I have posted this already somewhere in this giant thread :)

This will not be possible, unless you intend to do it manually for each author via some private message on O-F, as the forum's authentication is not exposed via an API, and it would not be practical to do so. We would likely need to use something like Okta token authentication, and the forum is not set up for that. And in any case, the forum here has no way to verify that all the authors on other repositories were, in fact, vetted properly. In short, it would require O-F to trust these other repositories that they manually verified user identity, and that is not practical, scalable, or reliable, even if well-intentioned.

Regarding this:
I have no intention of scraping anything from OF.

If that is truly the case, then there are no issues with your site accepting uploads and listing them however you wish. Currently the site is scraping from Orbiter-Forum, which is why @Xyon reached out to you via private message in the first place.
 
Status
Not open for further replies.
Top