segmentation violation on unix port

[uraich]

When trying to use the SDL driver on the unix port I get a segmentation violation:
import SDL
SDL.init()
crashes.

[embeddedt]

Do you have an Nvidia graphics card in your computer? The Unix port has issues with Nvidia cards which we haven't been able to track down. See #46.

[amirgon]

Hi @uraich !
Since we are unable to reproduce your problem on our side, we would need your help debugging this.
Could you provide the stack trace of the crash? You can obtain it by running Micropython under gdb. Something like gdb --args micropython ...

[uraich]

I think embeddedt gave the answer: I do have an NVidia graphics card

[amirgon]

I think embeddedt gave the answer: I do have an NVidia graphics card

@embeddedt Do you have an NVidia graphics card? Would you consider diving into this once again?

I can suggest the following:

• Run it with valgrind, perhaps there is some memory corruption
• Try to obtain the sources or at least the debug symbols of libnvidia-glcore and get a more meaningful stack trace than this one
• Try to ask on nvidia forums, or contact nvidia support
• Open a ticket on nvidia issue tracker
• Just for the test, we can try to change the SDL driver back to use SDL thread instead of Micropython thread. I believe the original problem was related to callbacks (we want to run Micropython callback on Micropython thread) so if this is the issue we can still use SDL thread but be carefully trigger callbacks from Micropython thread.

[uraich]

This is what I see when I run lv_micropython in gdb

[embeddedt]

Is this the same sequence of steps you ran to get it to segfault? It doesn't appear to have crashed yet.

[uraich]

Yes, the same sequence. Without gdb I see this:

[amirgon]

@uraich SIGUSR1 is used internally in lv_micropython and should be ignored.
Please run in gdb (before run):

handle SIGUSR1 nostop noprint pass

[uraich]

Correct!, So it is in the nvidia-glcore library

[embeddedt]

@amirgon Yes; I have an Nvidia card.

I've been reading some documents about SDL, and it appears that in order to be compliant with its requirements, we need to ensure that all SDL rendering is handled on our initial main thread. It appears that calling SDL functions from other threads is known to cause issues.

Is SDL always invoked from a specific thread, or can it be invoked by any thread depending on what MicroPython is doing?

[amirgon]

we need to ensure that all SDL rendering is handled on our initial main thread

@embeddedt Do you mean, from the same thread that initialized SDL?

Is SDL always invoked from a specific thread, or can it be invoked by any thread depending on what MicroPython is doing?

I think that SDL is initialized and rendered from the same thread all the time.

Here is how it works:

• mp_init_SDL is called from Micropython main thread when we call SDL.init() from Micropython.
  It calls monitor_init and initializes SDL.

lv_binding_micropython/driver/SDL/modSDL.c

Line 73 in 6e2af53

monitor_init(args[ARG_w].u_int, args[ARG_h].u_int);

• mp_init_SDL creates a new thread tick_thread, but this thread does not do the rendering directly. It only schedules a call to Micropython:

lv_binding_micropython/driver/SDL/modSDL.c

Lines 33 to 45 in 6e2af53

	STATIC int tick_thread(void * data)
	{
	(void)data;
	while(monitor_active()) {
	SDL_Delay(LV_TICK_RATE); /*Sleep for LV_TICK_RATE millisecond*/
	lv_tick_inc(LV_TICK_RATE); /*Tell LittelvGL that LV_TICK_RATE milliseconds were elapsed*/
	mp_sched_schedule((mp_obj_t)&mp_lv_task_handler_obj, mp_const_none);
	pthread_kill(mp_thread, SIGUSR1); // interrupt REPL blocking input. See handle_sigusr1
	}
	return 0;
	}

• When Micropython is ready it performs scheduled tasks and calls mp_lv_task_handler which performs LVGL and SDL rendering:

lv_binding_micropython/driver/SDL/modSDL.c

Lines 23 to 28 in 6e2af53

	STATIC mp_obj_t mp_lv_task_handler(mp_obj_t arg)
	{
	if (monitor_active()) monitor_sdl_refr_core();
	lv_task_handler();
	return mp_const_none;
	}

There is an open question here.

When Micropython performs scheduled tasks, is it doing it always from the same thread?
I think it is... but just to make sure it's worth adding some printing of Thread-ID to mp_lv_task_handler.

Looking at Micropython code, it's not entirely clear.
mp_handle_pending is the function in Micropython that runs scheduled tasks, but it is called in different places, specifically by MP_HAL_RETRY_SYSCALL which itself is also called in different places

[embeddedt]

Do you mean, from the same thread that initialized SDL?

Unfortunately, it's even stricter than that. It looks like SDL operations always need to be done on the initial main thread (i.e. the one which main(argc, argv) runs in at the start of the program). Doing them on a single thread consistently isn't enough.

Is the "Micropython main thread" the same thread as main(argc, argv), or does MicroPython spawn its own thread initially and use that for the rest of the program's lifetime?

[amirgon]

Is the "Micropython main thread" the same thread as main(argc, argv), or does MicroPython spawn its own thread initially and use that for the rest of the program's lifetime?

Looking at main.c I don't see any explicit creation of a new thread.
Also in the stack trace above it's clear that the call to SDL refresh is from the same thread main was invoked.

But to make sure, I suggest printing thread-id and checking if it's the same even when the problem happens.

[amirgon]

Another idea -
@uraich - Could you try running it with gdb again until it crashes, and show the stack trace of all threads?
We would be able to tell if there are other threads in the process and what they are doing.

gdb command:

thread apply all bt

[embeddedt]

No time to debug this right now, but assuming that Thread 1 is the main thread, it looks like we aren't violating any SDL requirements.

Thread 2 (Thread 0x7fffeffed700 (LWP 8085)):
#0  0x00007ffff7bc7c70 in __GI___nanosleep (
    requested_time=requested_time@entry=0x7fffeffece60, 
    remaining=remaining@entry=0x7fffeffece50)
    at ../sysdeps/unix/sysv/linux/nanosleep.c:28
#1  0x00007ffff71afad5 in SDL_Delay_REAL (ms=<optimized out>)
    at /tmp/SDL2-2.0.10/src/timer/unix/SDL_systimer.c:211
#2  0x0000555555653129 in ?? ()
#3  0x00007ffff71134ac in SDL_RunThread (data=0x555555d1d7e0)
    at /tmp/SDL2-2.0.10/src/thread/SDL_thread.c:283
#4  0x00007ffff71aa0a9 in RunThread (data=<optimized out>)
    at /tmp/SDL2-2.0.10/src/thread/pthread/SDL_systhread.c:79
#5  0x00007ffff7bbd6db in start_thread (arg=0x7fffeffed700)
    at pthread_create.c:463
#6  0x00007ffff6dc1a3f in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7ffff7faf740 (LWP 8081)):
#0  0x00007ffff1fe1447 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.450.66
#1  0x00007ffff1fe1ac3 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.450.66
--Type <RET> for more, q to quit, c to continue without paging--
#2  0x00007ffff1fb9c9e in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.450.66
#3  0x00007ffff1fc7e9b in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.450.66
#4  0x00007ffff1fd10dc in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.450.66
#5  0x00007ffff70f1d40 in GL_RunCommandQueue (renderer=0x555555c80d40, cmd=0x555555d1a550, vertices=0x555555d1a590, vertsize=<optimized out>) at /tmp/SDL2-2.0.10/src/render/opengl/SDL_render_gl.c:1270
#6  0x00007ffff70e9e11 in FlushRenderCommands (renderer=0x555555c80d40) at /tmp/SDL2-2.0.10/src/render/SDL_render.c:218
#7  SDL_RenderPresent_REAL (renderer=0x555555c80d40) at /tmp/SDL2-2.0.10/src/render/SDL_render.c:3130
#8  0x0000555555652f37 in ?? ()
#9  0x0000555555653169 in ?? ()
#10 0x00005555555b7954 in ?? ()
#11 0x00005555555b8d7a in ?? ()
#12 0x00005555555b8e90 in ?? ()
#13 0x00005555555d4d51 in ?? ()
#14 0x000055555565399a in ?? ()
#15 0x00005555555d4935 in ?? ()
#16 0x00007ffff6cc1b97 in __libc_start_main (main=0x5555555a537b <main>, argc=1, argv=0x7fffffffdbe8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdbd8)
    at ../csu/libc-start.c:310
#17 0x00005555555a53da in ?? ()