Testing Dashboard

[james-s-willis]

Hi,

I just read your SC paper (https://dl.acm.org/doi/10.1145/3731599.3767434) and I really like what you have done with your dashboard!

I would like to test the dashboard out on our OOD deployment and have gotten so far as to install it and I can launch the app:

In the paper you mention that "Common areas that need modifications include partition names, the cluster name, and cluster-specific
file system paths and scripts.". Can you list the files that I would need to modify to get the dashboard to work on our cluster?

Thanks for making this open source!

James

[richtan]

Hi James,

Thanks for checking out our dashboard! I've made a couple minor changes to the code that should fix some of the issues, so please make sure you have the latest version of the code. I've listed the sections that may need to be configured with your cluster-specific info/scripts below:

Component	Modifications
Announcements widget	Since at Purdue, we have a specific API that we fetch news from, this would require you to either have a similar API (see app/controllers/api/news_feed_controller.rb and app/views/widgets/_news_feed.erb) or provide the news data some other way. We are still working on making this more modular and simpler to plug other news sources into.
Recent Jobs widget	This widget shouldn't require any extra changes. Since you are getting an error, I can help figure out why it's not working if you post the relevant logs from the browser devtools and/or the Ruby on Rails server log.
System Status widget	Fill in the PARTITION_STATUS_INFO_URL, GPU_PARTITIONS, and EXCLUDED_PARTITIONS environment variables in your OOD env file.
Accounts widget	Modify the ACCOUNT_LIST_INFO_URL environment variable in your OOD env file.
Storage widget	We use a myquota script at Purdue to fetch info about users' disk usages. If your cluster has a similar script or data source for disk usage, you can substitute that in app/controllers/api/disk_usage_controller.rb. Also, modify the DISK_USAGE_INFO_URL environment variable in your OOD env file.
Homepage	The app/views/dashboard/index.html.erb file contains the layout for the homepage widgets. You can reorganize or disable widgets from here.
My Jobs app	None
Performance Metrics app	None
Cluster Status app	None
Job Overview	In the lib/util.rb file, modify the get_gpu_hours_usage function to align with your cluster's GPU hour usage calculation.
Node Overview	None

Other info:

• In the ARCHITECTURE_OVERVIEW.md file, it lists every component in the dashboard along with which files are associated.
• The environment variables should be set in the OOD environment file (i.e. /etc/ood/config/apps/dashboard/env).
• You can change the OOD_DASHBOARD_LOGO environment variable to be the file path to your cluster logo.
• The OOD_DASHBOARD_SUPPORT_URL and OOD_DASHBOARD_DOCS_URL environment variables control the support/user guide links in the navbar/sidebar.

If you have additional issues with any of the components, please let me know and I can help you get them working properly.

[james-s-willis]

Hi @richtan,

Thanks for the information. I have pulled the latest version of the dashboard now.

I thought it would be best to test one widget at a time, so I'll focus on the System Status Widget.

We use the submit_host option in OOD to issue SLURM commands on a remote host. So I changed app/controllers/api/partition_status_controller.rb to:

 5         partitions_output, partitions_status = Open3.capture2("ssh", "host", "sinfo", "-h", "-o", "%R|%a|%F|%C")                                                                                                                                                                     
 6         gpus_output, gpus_status = Open3.capture2("ssh", "host", "scontrol", "show", "node", "--oneliner")

where host can issue SLURM commands. I can issue these commands from the cmd line on our OOD portal server no problem:

ssh host 'sinfo -h -o "%R|%a|%F|%C"'
compute|up|6/56/0/62|248/4712/0/4960
compute_large_memory|up|2/1/0/3|44/196/0/240

However, when I come to start that widget I get:

Error fetching data: SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data [dashboard-install:199:13](https://OOD/pun/dev/dashboard-install)
    fetchWithCache https://OOD/pun/dev/dashboard-install:199
    load_partition_status https://OOD/pun/dev/dashboard-install:2866
    onclick https://OOD/pun/dev/dashboard-install:1

I also get this in logs/development.log:

Started GET "/pun/dev/dashboard-install/api/partition_status" for IP at 2026-01-12 10:37:04 -0500
Processing by Api::PartitionStatusController#get as */* 
No template found for Api::PartitionStatusController#get, rendering head :no_content
Completed 204 No Content in 2835ms (Allocations: 54870)

Would there be anything else to change for remote host submission?

Thanks!

[james-s-willis]

I was able to get the system status widget to work by changing the syntax in app/controllers/api/partition_status_controller.rb to:

 5         partitions_output, partitions_status = Open3.capture2("ssh ood073 'sinfo -h -o \"%R|%a|%F|%C\"'")                                                                                                                                                                     
 6         gpus_output, gpus_status = Open3.capture2("ssh trig-login01 scontrol show node --oneliner")

I also ran into an error related to excluded_partitions:

NoMethodError (undefined method `excluded_partitions' for class ActiveSupport::Configurable::Configuration):

which I was able to fix by prefixing Configuration.excluded_partitions.include?(p[:partition]) with :: to force the lookup of the function at the top-level.

[richtan]

That's great to hear! Apologies for not being able to help with that issue with the System Status widget, but hopefully I can help get the other widgets working.

For the Storage widget, where do users normally get information about their disk usage in the terminal on your cluster? Also, do you see any logs in the backend that can provide more details about the issue?

[james-s-willis]

I have managed to get the accounts info to display along with the recent jobs, by adding ssh host to the SLURM commands.

The storage widget is reporting:

Started GET "/pun/dev/dashboard-install/api/disk_usage" for 216.73.177.50 at 2026-02-02 15:00:37 -0500                                                                                                                                                                                    
Processing by Api::DiskUsageController#get as */*
Completed 500 Internal Server Error in 77ms (Allocations: 58946)

Errno::ENOENT (No such file or directory - myquota):
  
app/controllers/api/disk_usage_controller.rb:5:in ``'
app/controllers/api/disk_usage_controller.rb:5:in `block in get' 
app/controllers/api/disk_usage_controller.rb:4:in `get'

but that is because we don't have the myquota command in our cluster. We have a diskUsage command that shows users their storage usage:

diskUsage 
                            Description                Space         # of files
                    /home (user )        84GiB/ 100GiB         452K/1000K
                 /scratch (user )      7493GiB/  25TiB         313K/  10M
              /project (project )       907GiB/5120GiB         450K/  10M
      /project (project ab)         0  B/1024GiB           3 /2000K
         /project (project 2)         0  B/1024GiB           5 /2000K

I have already set up disk quota warnings on our dashboard: https://osc.github.io/ood-documentation/latest/customizations.html#individual-user-quota
Is it possible to reuse this .json file to populate the storage widget?

[richtan]

Unfortunately, I don't have it setup so that it can read from the .json file that the disk quota warning uses, but I can make that addition to the repo. It also seems like your diskUsage command is quite similar to our myquota command's output, so that should also work with just slight modifications to the parsing of the command output.

[james-s-willis]

Can you show me an example of what the expected format is for:

{ type: s[0], location: s[1], disk_usage: s[2], disk_usage_limit: s[3], disk_usage_percentage: s[4], file_count: s[5], file_count_limit: s[6], file_count_percentage: s[7] }
        }.to_json

or an example of what your myquota command returns.

[richtan]

This is a sample output from our myquota command:

Type       Location             Size    Limit    Use   Files   Limit    Use
===========================================================================
home       x-user             17.0GB   25.0GB  68.2%      -       -      -
scratch    x-user              0.0KB  100.0TB   0.0%      0     1.0M   0.0%
projects   x-abc070002         4.7TB    5.0TB  94.2%  604.2K    2.1M  28.8%
projects   x-def270241        12.1TB   30.0TB  40.3%    9.3M   18.4M  50.5%
projects   x-ghi260214        43.2GB    3.0TB   1.4%   22.5K    1.0M   2.3%

That should return the following JSON in app/controllers/api/disk_usage_controller.rb:

[
  {"type":"home","location":"x-user","disk_usage":"17.0GB","disk_usage_limit":"25.0GB","disk_usage_percentage":"68.2%","file_count":"-","file_count_limit":"-","file_count_percentage":"-"},
  {"type":"scratch","location":"x-user","disk_usage":"0.0KB","disk_usage_limit":"100.0TB","disk_usage_percentage":"0.0%","file_count":"0","file_count_limit":"1.0M","file_count_percentage":"0.0%"},
  {"type":"projects","location":"x-abc070002","disk_usage":"4.7TB","disk_usage_limit":"5.0TB","disk_usage_percentage":"94.2%","file_count":"604.2K","file_count_limit":"2.1M","file_count_percentage":"28.8%"},
  {"type":"projects","location":"x-def270241","disk_usage":"12.1TB","disk_usage_limit":"30.0TB","disk_usage_percentage":"40.3%","file_count":"9.3M","file_count_limit":"18.4M","file_count_percentage":"50.5%"},
  {"type":"projects","location":"x-ghi260214","disk_usage":"43.2GB","disk_usage_limit":"3.0TB","disk_usage_percentage":"1.4%","file_count":"22.5K","file_count_limit":"1.0M","file_count_percentage":"2.3%"}
]

[james-s-willis]

Thanks for your help, I have it working now!