Skip to content

Lin Thread Queue test reliability #363

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jmid opened this issue Jun 8, 2023 · 0 comments · Fixed by #411
Closed

Lin Thread Queue test reliability #363

jmid opened this issue Jun 8, 2023 · 0 comments · Fixed by #411
Labels
test suite reliability Issue concerns tests that should behave more predictably

Comments

@jmid
Copy link
Collaborator

jmid commented Jun 8, 2023

I've seen the Lin Thread Queue test find a counterexample under Windows bytecode in less that 250 attempts more than once.

This happened yesterday on 5.1.0~alpha2 windows bytecode:
https://github.com/ocaml-multicore/multicoretests/actions/runs/5199454489/jobs/9376867328

random seed: 189508584
generated error fail pass / total     time test name

[ ]    0    0    0    0 / 1000     0.0s Lin DSL Queue test with Domain
[ ]    0    0    0    0 / 1000     0.0s Lin DSL Queue test with Domain (generating)
[✓]    2    0    1    1 / 1000     5.1s Lin DSL Queue test with Domain

[ ]    0    0    0    0 /  250     0.0s Lin DSL Queue test with Thread
[✗]   68    0    1   67 /  250    16.8s Lin DSL Queue test with Thread

--- Info -----------------------------------------------------------------------

[...]

--- Failure --------------------------------------------------------------------

Test Lin DSL Queue test with Thread failed (0 shrink steps):

[...]

  Results incompatible with sequential execution

                                               |                    
                           Queue.take t : Error (Stdlib.Queue.Empty)
                                    Queue.take_opt t : None         
                                    Queue.is_empty t : true         
                                    Queue.is_empty t : true         
                                    Queue.is_empty t : true         
                           Queue.take t : Error (Stdlib.Queue.Empty)
                                      Queue.clear t : ()            
                                               |                    
                         .------------------------------------------.
                         |                                          |                    
                Queue.length t : 1                         Queue.add 7 t : ()            
            Queue.take_opt t : Some (7)                Queue.peek_opt t : Some (7)       
                Queue.length t : 1                         Queue.add 1 t : ()            
               Queue.peek t : Ok (4)                       Queue.add 4 t : ()            
               Queue.peek t : Ok (4)                      Queue.peek t : Ok (4)          
              Queue.is_empty t : true                     Queue.peek t : Ok (4)          
     Queue.take t : Error (Stdlib.Queue.Empty)         Queue.take_opt t : Some (4)       
                Queue.clear t : ()                       Queue.take_opt t : None         
                                                           Queue.clear t : ()            

================================================================================
failure (1 tests failed, 0 tests errored, ran 2 tests)

And earlier

@jmid jmid added the test suite reliability Issue concerns tests that should behave more predictably label Jul 14, 2023
@jmid jmid closed this as completed in #411 Nov 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test suite reliability Issue concerns tests that should behave more predictably
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant