I'm not sure if it's overkill but decapCMS [0] (formerly netlifyCMS) uses git for its publishing method to store static files and content. It's quite simple and open source, and might even just serve as good inspiration for your final solution.
If it's JSON it sounds more like config than code. Does it actually need to live in the GitHub repo? Could your application read it from an external location (where ever the rest of your configs live) and reload it on an API call or periodically?
We tried this but the performance hit was too high
* edit: we're building an integration with a SaaS product that doesn't allow us to reference external files due to security, hence having to upload direct to git
I mean, we're talking about git here - so you don't need to re-invent pushing a file to the repo. What you need to focus on is building a UX that abstracts it so the non-tech folk don't need to do it via command line.
So the question is less about pushing to git and more about what causes the file to update in the first place. If it is an automated update, just have whatever task updates it also run the git command to push it. If it is a manual update, what tool are the non-tech people using that makes it happen? Add a button to that tool which runs the command.
I wrote an ansible pipeline that pushes config backups, json and ndjson, to a GitHub repo. It just uses the “git” cli tool to clone the repo, commit any changes, and push those commits to “origin/main”.
Only piece that you have to be a bit careful with the guarding credentials or token you will be using for the “git” commands. In my case we use Hashicorp Vault for secrets management so I can just checkout the token to use from there.
You can do this in any number of ways. Powershell, batch script, bash, you could even build a little GUI if you want.
If you want to get fancier, throw the script on a server and give users an SSH script they can double-click, or hook it up to some intranet web interface.
This isn't a particularly complex problem, and git was designed to handle use cases just like this.
Of course you could absolutely overcomplicate and burn dozens of engineer hours building some kubernetes cluster or AWS swarm, but you can do this with a script so simple that I wrote it on my phone.
I'm not sure if it's overkill but decapCMS [0] (formerly netlifyCMS) uses git for its publishing method to store static files and content. It's quite simple and open source, and might even just serve as good inspiration for your final solution.
[0] - https://github.com/decaporg/decap-cms
Just use gist files API.
https://docs.github.com/en/rest/gists/gists?apiVersion=2022-...
Interesting, I'll give that a shot
If it's JSON it sounds more like config than code. Does it actually need to live in the GitHub repo? Could your application read it from an external location (where ever the rest of your configs live) and reload it on an API call or periodically?
We tried this but the performance hit was too high
* edit: we're building an integration with a SaaS product that doesn't allow us to reference external files due to security, hence having to upload direct to git
I mean, we're talking about git here - so you don't need to re-invent pushing a file to the repo. What you need to focus on is building a UX that abstracts it so the non-tech folk don't need to do it via command line.
So the question is less about pushing to git and more about what causes the file to update in the first place. If it is an automated update, just have whatever task updates it also run the git command to push it. If it is a manual update, what tool are the non-tech people using that makes it happen? Add a button to that tool which runs the command.
Agreed - I'm new to git so the question is what code that button is firing? Ideally in a cloud app not desktop app.
I wrote an ansible pipeline that pushes config backups, json and ndjson, to a GitHub repo. It just uses the “git” cli tool to clone the repo, commit any changes, and push those commits to “origin/main”.
Only piece that you have to be a bit careful with the guarding credentials or token you will be using for the “git” commands. In my case we use Hashicorp Vault for secrets management so I can just checkout the token to use from there.
#!/bin/bash
commandToGenerateFile > file.json #or wget some.server.tld/autogen.json file.json git add file.json git commit -m "Automatic upload <date>" git push
You can do this in any number of ways. Powershell, batch script, bash, you could even build a little GUI if you want.
If you want to get fancier, throw the script on a server and give users an SSH script they can double-click, or hook it up to some intranet web interface.
This isn't a particularly complex problem, and git was designed to handle use cases just like this.
Of course you could absolutely overcomplicate and burn dozens of engineer hours building some kubernetes cluster or AWS swarm, but you can do this with a script so simple that I wrote it on my phone.
yeah running a bash script is one option
the k8 approach is tempting though haha