Active force direction does not update correctly on GPU (GPU P100)

[joy-2boy]

Description

When running a simple active Brownian dynamics simulation, the direction of the active force stops updating after approximately 200 steps only when using the GPU. The simulation behaves as expected on the CPU.

Script

import hoomd
import numpy as np
import time
import gsd.hoomd
import os

# Select GPU device
os.environ["CUDA_VISIBLE_DEVICES"] = "0"  # Change as appropriate

# Simulation parameters
seed = 42
num_particles = 1
active_force = 1e-1
delta_t = 1e-2
steps = 500
dump_period = 1
box_length = 10
box_size = [box_length, box_length, 0]

# Initialize particle at origin
position = np.zeros((num_particles, 3))
position[0] = [0.0, 0.0, 0.0]

# Create snapshot
snapshot = gsd.hoomd.Frame()
snapshot.particles.N = num_particles
snapshot.particles.position = position
snapshot.particles.typeid = [0]
snapshot.particles.types = ['A']
snapshot.configuration.box = [box_size[0], box_size[1], box_size[2], 0, 0, 0]

# Save initial configuration
with gsd.hoomd.open(name='initial_single.gsd', mode='w') as f:
    f.append(snapshot)

# Initialize simulation
device = hoomd.device.GPU()  # Replace with hoomd.device.CPU() to observe expected behavior
sim = hoomd.Simulation(device=device, seed=seed)
sim.create_state_from_gsd(filename='initial_single.gsd')

# Apply active force
active = hoomd.md.force.Active(filter=hoomd.filter.All())
active.active_force['A'] = (active_force, 0, 0)

# Rotational diffusion updater
rotational_diffusion_updater = active.create_diffusion_updater(
    trigger=hoomd.trigger.Periodic(1), rotational_diffusion=0.1)
sim.operations.updaters.append(rotational_diffusion_updater)

# Brownian dynamics
brownian = hoomd.md.methods.Brownian(filter=hoomd.filter.All(), kT=0)
integrator = hoomd.md.Integrator(dt=delta_t, methods=[brownian], forces=[active])
sim.operations.integrator = integrator

# Output
gsd_writer = hoomd.write.GSD(
    filename='trajectory_single.gsd',
    trigger=hoomd.trigger.Periodic(period=dump_period),
    mode='wb',
    dynamic=["property", "momentum"]
)
sim.operations.writers.append(gsd_writer)

# Run
print("Simulation start!")
start_time = time.time()
sim.run(steps)
end_time = time.time()
print(f"Simulation complete. Duration: {end_time - start_time:.2f} seconds.")

Input files

No response

Output

Simulation start!
Simulation complete! Time elapsed: 0.07 seconds.

(However, upon inspecting the trajectory_single.gsd file, I found that the particle's orientation quaternion stops updating after a certain number of steps.)

Expected output

No response

Platform

GPU

Installation method

Compiled from source

HOOMD-blue version

5.1.1

Python version

3.10.17

[joaander]

Thanks for the report and complete minimal reproducible example. I added:

gsd_writer.flush()

trajectory = gsd.hoomd.open('trajectory_single.gsd')
for frame in trajectory:
    print(frame.particles.orientation[0])

to the end of your script and then executed it.

Unfortunately, I am not able to reproduce the behavior you describe on a V100. I see the particle's orientation change on every step. Here are the last few lines of output:

[ 0.50563705  0.          0.         -0.8627463 ]
[ 0.5183843  0.         0.        -0.8551478]
[ 0.53454274  0.          0.         -0.8451414 ]
[ 0.5629025  0.         0.        -0.8265233]
[ 0.5517166   0.          0.         -0.83403164]
[ 0.5626785  0.         0.        -0.8266758]
[ 0.5572058  0.         0.        -0.8303744]
[ 0.53705883  0.          0.         -0.8435448 ]
[ 0.5639268   0.          0.         -0.82582474]

I do not have access to a P100 for testing. Please try the following to troubleshoot potential problems:

1. Add device.gpu_error_checking = True after constructing the device.
2. Run compute-sanitizer --tool memcheck python3 script.py
3. Run compute-sanitizer --tool racecheck python3 script.py
4. Run compute-sanitizer --tool synccheck python3 script.py

All of these checks pass with no errors when I run your script.