Did development start yet?

>>11014
In a way it's been started for a while.
Software languages(Golang only), libraries (Gin, Pongo2, MySQL) and development process established(Test heavy/driven development).
Use case list needs to be written so I can't start the code yet.
Then I have to make some architecture related decisions which will decide where all the code files go.

So still in pre-alpha. Multiple "prototypes" have been written. I might want to do something else to get a few design patterns understood, but probably not necessary and can just wing it.

Is it fun to write an imageboard engine?

I wouldn't call it super fun, but I enjoy rigging things up on the backend. I loathe how webdev is these days so I enjoy finding clever solutions to problems on that end too where most would just use JavaScript and 700 libraries instead of clever HTML and CSS hacks.

>>11015
Why not use more of the standard library? Go houses some powerful features right out of the box, including a HTTP server and a HTML templating engine. It's two less dependencies you have to worry about.
I know net/http and html/template can be cumbersome at points and getting the initial handler/templating layout can be a little hard at first but it works damn well once you get it to work.
Then again, I've only ever used that and the Fiber framework.

How are you designing the engine? Doing database stuff first and testing where possible, then tossing everything else on top?

>>11021
Gin looks to be similar to Fiber.
Gin and Pongo2 has lots of usage in major websites. Only issue I've come across is that fuzz testing with Gin is very inefficient. Just can't do it without re initializing the server every round. So basically I have to abandon one of the modern aspects of Golang's standard testing library. Why not use the standard net/http, just because I've tried doing that with the Kissu's server which generates the JSON pages(amoung other things) and it ended up being a mess. Not going to reinvent the wheel.
As for Fiber. I don't like the Express engine it's based on(it's what serves these pages to you and generates the JSON which client side renders these pages). Something about the handlers didn't stick with me.

html/template is fine. It's alright, but I might as well go big. Also I can't find any information about using custom functions inside of templates with html/template while Pongo2 allows this. Pongo2 and Vichan's Twig are similar in that they use functions inside of templates regularly, but yet it will still require a bit of effort to translate between the two. There is a Golang Twig analog called Stick and I used that in the feedback engine, but it's lacking in support(still not a finished product) and studies. Some other alternatives such as Amber are a complete departure from Twig's syntax.
https://medium.com/@kataras/whats-the-fastest-template-engine-in-go-fdf0cb95899b
Blocks is not very active but an alternative I guess.

I said, pretty clearly that I'm not going to be amateurishly hacking everything on top. Pretty rigid adherence to the practice of TDD. Maybe the routing will be done with manual testing.
Database will still be MySQL, but I figure it's time to get rid of posts_b, posts_jp(and it's files json entry attatched to each post) and start a more by-the-book relational database instead of the existing no-sql and relational hybrid.

The order by which I'll be doing things I'm not sure about. I need to build up what will likely be a 100+ item use-case list covering things from Vichan and my support servers which will be integrated into this future imageboard engine. Then I will write up some architecture diagrams, somewhat resembling UML but not really, just a way for me to collect my thoughts.
When all those are done, I begin writting to fill in the blanks until I have a Beta product which I'll put onto the site to do active testing.
This is the system that I did to create the UI, but this time it will be after having experienced issues with my development process on the UI and a more professional take on understanding what I'm getting into before I begin writing. More planning, less hacking.

>>11022
Fiber is good for getting things up and going fast, but I agree with your criticisms. I don't hate it, but I wouldn't use it for anything serious that I would start today; I have one project in it which works, but I occasionally found myself beating my head against the wall because of Fiber.
net/http is barebones but you can build your functionality on it at least and only include what you need. My most major project in it uses regular expressions to determine which route to take, but it also has only three routes anyway.

>using custom functions inside of templates with html/template
You can define functions in Go and use them quite easily; allow your template to use them by tossing them into (*Template).Funcs, functions that have a return type of template.HTML will not be HTML escaped but all others will.
Inside of templates is a bit weirder and as such I don't really bother with it, but essentially {{define "template name"}}do stuff with {{.}}{{end}} and call with {{template "template name"}} or if you want to set dot, {{template "template name" <dot value>}}.
However, by the time I'm doing that, I'm hitting the absolute limits of html/template anyway so I just write the same thing in Go, where it would be faster regardless of the templating engine, except quicktemplate which is a different beast altogether.

>performance
Don't worry about super fast performance until you're getting those numbers of requests per second, in which case you're likely getting hit with an attack anyway.
I would assume that most would not notice the few hundred microseconds on a slightly faster engine either.

>I figure it's time to get rid of posts_*
You should keep them to reduce query complexity, but I would move the files JSON into the database. Unless it already is, in which case I now wonder why you would toss JSON into the database in the first place, but I digress.
My imageboard projects never made it outside of my machine (save one but it's a textboard), but I had no qualms with a {posts,files,replies}_board table; don't take this from me however, as I am by no means an expert on anything I do, nor is everything I do right and best.

>>11023
>(*Template).Funcs
ah, i see now.

No, I'll worry about performance and making it work quickly for as long as the site expands. This isn't slight either, it's double as much throughput. The only argument you can make is that Golang with make a PHP-esque change and fuck up every dependency that relies on it.

Files is stored as json inside of the posts_* entries.
Increased query complexity is important because doing lookups to build /all/ is monstrously expensive because I have to write 9 different union statements for each board.

This Funcs system is pretty ugly.
https://www.calhoun.io/intro-to-templates-p3-functions/

Compare it to this pongo2 example.
https://github.com/ECHibiki/Kissu-Feedback-and-Forms/blob/main/src/server/templater/pagegen.go
I prefer this system more. Assigning functions to the template engine when you init is better than doing it at runtime, yea

>>11024
>Increased query complexity is important because doing lookups to build /all/ is monstrously expensive because I have to write 9 different union statements for each board.
If you're set on getting rid of the several boards, I recommend you check out the source to Picochan which does just this. I can upload if you so wish. It's a Lua CGI SQLite3 onion imageboard that I do not frequent yet have the source of, which I have combed through enough to tell you that this one table scheme what it does.

I figure it would be monstrously expensive for the overboard regardless of what you do, but I'll dump in my suggestion regardless: I wonder what the performance would be if you were to do it automatically.
Create a table housing a thread's ID, its board, and its last bump. A trigger automatically creates, updates, and deletes from this table as needed.
Pull from this table and sort by the last bump date and gather only what you need with pagination.

I may try this myself with some dummy data soon for my own projects.
I'll sleep before I start sounding more like a fool, thank you for your time.

>>11026
I don't know why you'd suggest all this stuff when a site like pornhub literally runs on a MySQL fork(wish I remembered which one) and does fine.

Another point that has come up where posts_b, posts_jp is causing issues...
When a post is moved between boards, or just moved in general, it requires a recreation of the existing thread and then a deletion of the old one.

With a new system it only requires duplication(in the case of cloned moves) or a change in a datafield with where it belongs.

There is very little reason to have there be board specific database tables other than the thought that it will make sites go a little bit faster on posts(of which image manipulation is the most computationally expensive task anyways...)

I'm going through all of Vichan and my own stuff's functions and writing out use-case lists which I'll use during my architecture formulation stage.

This use-case list will be open source, basically encapsulating all of kissu-vi's functionality and things I plan to add to Tsukuyomi
https://github.com/ECHibiki/Tsukuyomi-UseCase-List

I'm writting into my draft every time there is a delete action that they will be stored in an inaccessible recycling bin for a few hours. This will allow for moderator controlled rollbacks on some of our actions which are normally irreversible.

These would be true deletes(user deletes), page renumbering deletes, archive deletes, cyclical deletes.

A proposal I have to add in a long term text archive of kissu would be connected to the concept where some of these recycling bin items that are cleared from recycling would be accessible on another database. This would be a lightweight version of the archives we know well on 4chan, but on Kissu.
Ghostposting you might ask? We'll think about it.

Finally starting on graphing out the program.

One of the more advanced things which I don't think any other imageboard software does is security checks on startup.
Tsukuyomi will:
- Check if files and folders are at safe permission levels (Nothing should be root 777)
- Check that settings files are not accidentally being read by NGINX file rules

>>11121
What program is that? Looks pretty useful

>>11122
GitMind. https://gitmind.com/
Very general purpose program.

When I was initially drafting the UI I used some terrible thing which we used in university. This is leagues better

I don't think I actually have the graphs on local storage though. Haven't checked. Seems like cloud data.

https://gitmind.com/app/docs/flwp22tg

I'm considering potential ways to use this trick
>>>/qa/101227

one optimization coming to mind is that this can be used on thumbnails to reduce storage in the archive concept I want to add.
Another is that post times could be increased greatly by evaluating a blockhash of a submitted image... such that there's no need for image processing...

Very interesting trick

Almost at the first draft of the diagram stage.
Have to describe what each of these SQL tables will be doing and their FK associations

Effectively our first draft.

Won't make any sense to anyone but me yet.
My next task is to go correct issues, expand missing details, arrange the elements.

Following that will be to write out the pseudo code... text based summary of the operations... and following a similar process as the diagram stage.

Following this... the conversion of pseudo code into software(writing out code from text descriptions) is getting to a point where AIs can assist programmers to create buggy, insecure and unmaintainable code.

A copy of my draft is: https://gitmind.com/app/docs/f2r64yan
Will be my documentation only github shortly: https://github.com/ECHibiki/Tsukuyomi-UseCase-List

hm, for some reason it didn't add labels to the SVG export.

What a modular abomination

Progress on the rewiring and finalization of the blueprint.
After a break will do the first line of packages which will complete 60% of it.

A bit neater.

Think I will have to divide up one of the sections into a more detailed view. Or maybe not. I want to get into pseudo coding by the New Year so I might fill in some blanks as I go.

There's more that I could do with this, but it would get to the point where it's getting a bit obsessive.

https://gitmind.com/app/docs/f6pvzikf

I'll just leave it at this for now

Christmas is over. back to work tommorow