Client side appliation and frameworks?

[MikeNeilson]

@opendcs/opendcs-core-devs, but specifically @wjonassen, @agilmore2 , @adamkorynta , @DanielTOsborne, and @krowvin

So the current Web Client is primarily JSP, and thus a lot of server side rendering. Nothing fundementally wrong with this but there's definitely some pros and cons to consider in regards to single page applications vs combined server side.

I'm starting in on the DDS over HTTP(s), which will include a web based version of the "rtstat" program. @krowvin intermittently pokes at revamping DcpMon; which likely just be part of that web based rtstat going forward.

I'm inclined towards setting up the LRGS system as a "single page app", or at least more focused on client side rendering using one of those frameworks.

USACE is pretty well bought into React as the framework, and while I have some issues with it I'm inclined to use it for that reason plus the fact that I got a large bundle of training off of humble bundle a while ago. It does also allow the use of typescript which can be rather handy.

Does anyone have any thoughts on an appropriate javascript framework for developing these apps?

e.g.

• React
• Other
• Just do vanilla javascript and call it a day
• Don't do SPA's and here's some docs as to why?

I'm not hard set on either direction, but it would be good to settle on a direction to start.

[DanielTOsborne]

Is there a way to get equivalent status information without the web page?

Despite USACE's current preference towards react and client-side rendering, I don't like it because it makes the result unparseable by automated systems (some may see that as a pro, but for our mission, access to data without obstacles is vitally important).

[MikeNeilson]

Yeah, my plan of implementation would be an api that likely initially just provides the plain XML that report is based on but then over time breaking it up into separate endpoints to provide more flexibility.

It wouldn't be one or the other. The idea with the webclient is to add functionally and remove the need for RtStat itself. I'm totally with you being more flexible so that if a use doesn't want our default, they don't need to use it.

[krowvin]

I think we should have both.

Have a React frontend for those that like that/endusers.
Have static pages for automated systems or other entities that prefer plaintext

Create a robots.txt and sitemap.xml to let the bots and users know where to go for what

[MikeNeilson]

I think a simple API vs a static page is better. Scraping HTML is a royal pain. And the lrgsstatus page is derived from XML. For the client side front end I would just have the XML output, and consume it. So anyone that wanted to scrape or make their own page would be able to do so.

This discussion is for how OpenDCS, by default, will render and maintain rich content. Just to be clear we're going to support both, but what does OpenDCS itself choose to do.

FYI, I'm going well beyond the status page with this idea, the LRGS is always configured over network so we don't need a thick client, we can get away with a website that the admin can interact with. It's the configuration and user management I'm considered about at the moment.

[krowvin]

I am partial to using React + Vite for the frontend framework.

Pros

• The primary reason is we can use "Groundwork" which is a React component library that WM and other USACE entities are starting to use
• in order to be 508 Compliant (Accessibility)
• This offloads that work to the maintainers of this framework
• We could potentially put some of these components in the Groundwork-Water component library
• Offers linting / Better "runtime/browser" checking of the pages
• SPA could also be converted to a PWA (progressive web app) which allows offline use to some degree

Cons

• Poor for web scraping purposes - but perhaps we add links to the API
• Adds a potential other level of build steps that we would need (react (or typescript) gets transpiled to javascript)
• Learning curve - JavaScript is the basis of react. More almost certainly know it.
• React could change or go away, JavaScript is eternal

I will also say it does not need to be all or nothing.

Where it makes sense we could still have endpoints that return static and/or Server Side Rendered (SSR) HTML/txt.

Thoughts on where to start

Our district at USACE (SWT) has a major investment in this due to looming cloud migration.

We need a replacement for our page here:
https://www.swt-wc.usace.army.mil/dcpstatus/

This uses getDcpMessages the opendcs program that reads the archive and writes it to a large dump file. We then parse this and write various json files to build the above webpage.

The API and/or webpage that results from this could serve as that replacement.

Here are some sample json files of what we convert the output of getDCPMessages to:
(not saying it needs to be done this way)
Index File

KEYS Gage File

[MikeNeilson]

If you're talking to an LRGS, if it's offline, it's not working... though the current RtStat app does support connecting to multiple LRGSs so well, you might have something there actually.

[MikeNeilson]

As for ground work, I don't see a compelling reason to embedded that in to the OpenDCS project, though perhaps it's more generic than I'm thinking.

The branding for this discussion will all be "OpenDCS", though it likely makes sense for a user to be able to customize the branding and logos appropriately.

[krowvin]

If you're talking to an LRGS, if it's offline, it's not working..

The idea of a PWA and having the ability to be in offline is that a service worker on the webpage will still load if the server that returns the HTML itself goes down.

This would enable you to make requests to the LRGS API, or another, because the source html would still be on your system.

It's more about the server serving the html being down vs the API itself.

You'd just handle api errors if the rest api was down.

https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Tutorials/js13kGames/Offline_Service_workers

But you get service workers with vanilla JS too. So really i was just commenting on your point about going for a SPA. This same single page app could be a PWA.

The branding for this discussion will all be "OpenDCS", though it likely makes sense for a user to be able to customize the branding and logos appropriately.

Ah yes, good point. So really groundwork doesn't matter in this context. We would need to make our own branding components.

[MikeNeilson]

If you're talking to an LRGS, if it's offline, it's not working..

The idea of a PWA and having the ability to be in offline is that a service worker on the webpage will still load if the server that returns the HTML itself goes down.

This would enable you to make requests to the LRGS API, or another, because the source html would still be on your system.

It's more about the server serving the html being down vs the API itself.

You'd just handle api errors if the rest api was down.

https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Tutorials/js13kGames/Offline_Service_workers

This idea I like, and may allow us to keep the current behavior of load "app" and point to different LRGS systems.

[MikeNeilson]

Something for everyone to remember in this discussion. The OpenDCS applications, web or others, MUST be able to operate on their own without regards to other implementations like, USACE's cloud migration. It is independent software. This means there will likely be some amount of duplication due to branding issues and the like.

[MikeNeilson]

Put another why. The question is about general OpenDCS Modernization. So like definitely good to point out current short comings like the limited status page, but the focus is on general OpenDCS Modernization, not any users specific cloud migration efforts.

The USACE/React mention was to describe the availability of knowledge.

[krowvin]

Yeah good point - you'll have to remind me again, i'm sure, that OpenDCS is truly it's own thing and even using the same color as USACE does not make sense.

[MikeNeilson]

Upon review of various Javascript GUI frameworks, and having done html/js guis from scratch in the past,for the LRGS web interface I'm going to just go with React.

NOTE for the one dissenter above. There will be a full featured API to retrieve appropriate data the this framework would interact with. I agree we don't want something like React to "dominate and be 'the thing'" but we do want to present a nice default UI to the end user that is both easy to use and maintain. Say what you will about React (I also have concerns) but Charles is right that if I went down plain JS I'd just be making my own framework to maintain.

If the general web api goes a different way and it looks easier to maintain I'll suffer the effort of porting it when the time comes.

[krowvin]

What are the developers of the OpenDCS interface using for the framework on the frontend?

It might be worth using the same as them so we can borrow components from each other or make a shared library.

But that may be only for the branding given the differences between the LRGS and all that CCP can do?

[MikeNeilson]

current jsp, html, css, js. Not a single page app. so basically nothing.
plus i'm not really sure how much would be sharable in that context.

but that is why this question is on the rest_api repo.

[wjonassen]

It would definitely be easier to move to vanillla javascript and call it a day, quicker and less time to do the conversion.

With React, myconcern is that it is a technology that could over time get less popular and fall out of grace, similar to angularjs. I understand it's very popular though, and is going strong right now, and supported by large companies like facebook, so it's got a lot of staying power for sure, you just can't predict the future. Also, the corps is using groundwork-water, which could be a benefit. First off - people will have the experience in React, so that's good. Also, if we feel that groundwork-water is going to stick, or already has, then it might make sense to go in that direction, but if we want it to truly independent, then maybe using groundwork-water would couple it too closely to USACE - not sure on this one though, just my thoughts.

[MikeNeilson]

I don't think, for OpenDCS, groundwork-water is the way to go. No a knock on groundwork but it is usace and the cwms data api focused, so would just get in the way.

As for React vs Vanilla. I definitely agree on the concern; however, it has stayed around a while. And if you really think about it, if we want all this to actually be maintainable, we're going to just end up writing another framework anyways.

That said, we do make our own decisions here. So while @krowvin and I are doing React for the DCP Monitor web-based-LRGS, if you would prefer vanilla for the GUI I don't think it unreasonable for us to try both at the same time and compare notes.

Just consider the level of complexity you're managing, we do want anyone qualified in web development to be able to pick up and work on this -- whether they actually do or not. So it needs to be structured in such a way that should we alter some fundamental concept of a component we can properly isolate the work.

I and others will of course keep an eye on work and help identify areas where there's too much duplication or where things can be generic though.

Also keep in mind we would like to be able to leverage the Web GUI for the "desktop" using electron or Tauri. Single Page Apps (whether React, vanilla, or other) make that easier. E.g. the launcher (which would never really be on the web) could be a single page app that spawn specific single page apps. (in the case of the current GUI I feel like it would be "the app" but per profile.) - I hope that didn't add confusion.

[MikeNeilson]

Quick bump on this discussion. Reviewing the JSP and existing Javascript there's just... a lot, and some of it repeated, and it does seem like it's going to be easy to maintain.

However, it does have a reasonably good layout that I think we could keep, at least until someone pitches something we all agree is better.

That said, if the goal is to put the effort into rebuilding to something maintainable. What do we actually want to do? I think there's a few of us that would like to get started on that. I've got the opendcs-operator for k8s coming along reasonable well and will be using it soon and USACE is will need something for our cloud migration.

So question again do we:

1. Continue JSP + Javascript... but we look at what we have, diagram things out and reorg given current knowledge
2. Migrate to React
3. Use another tool

For LRGS Charles has already gone down the React route, which will mean integrating a React project into the main code base at the same time or rather soon after the rest_api/web_gui is.

JSP itself, isn't drastic to include, it's just another gradle subproject of "java". Another tool;however, would require additional build integration methods that we have to maintain. Which makes me think we should just go React, but I really don't want to be the guy that just makes all of these decisions like a dictator.

I'm also quite certain that this decision is going to affect some upcoming contract work; however, to be blunt, the project can't let the existence of a contract dictate it's direction.

@opendcs/opendcs-core-devs @opendcs/opendcs-triage

[MikeNeilson]

So far I've only heard from @krowvin which was in support of sticking with JSP, and pointing out the JSP fragment system.

I'll give it until the end of the week but hearing nothing I will take that as the path forward. Given our varied implementations that is likely the better path forward. I've made plugins for TeamCity that implement additional gui behavior by providing appropriate HTML/JSP files in the plugin jar; I suspect we'll find ourselves needing something similar before too long.

That said, I will likely engage in a major reorg. We want to get any used dependencies within autoupdatable infrastructure and not store and manually maintain them in the repo; fortunately the "webjars" concept exists so should be fairly straightforward.

[ktarbet]

I'm interested in a clean separation between the UI and API.

Building a client with JavaScript/React , with modern browsers would be a way to force that separation.

[krowvin]

@krowvin which was in support of sticking with JSP

I should say, I meant for now

But then you must ask, do you continue to make new pages/components in JSP (and resolving issues). Or do you start those in a new branch with a React update for the UI? (Assuming we go that route)

[MikeNeilson]

Given we've had little input, and to have some allowance for upcoming work, for the moment the direction is going to be stick with the current JSP setup. At least until we've determined it isn't going to be practical long term. I would like us all to continously think about that as we do any additional development.

@krowvin has voluntereed to attempt correct some dialog behavior.
I will update the login page to support selecting what identity provider to use.

Additionally I am currently cleaning up the javascript dependency management. https://webjars.org is a useful place to integrate a javascript library into a Java/JSP project and should be the primary place we retrieve dependencies if we aren't using npm. In-lieu of that please ensure that dependencies are retrieved in a way that is trackable for updates and avoid pulling things in just to pull them in.

That said, it's also not impossible to just include NPM into the build for some additional dependency management if we determine that is more practical.