Let's Write a Qi Compiler! - #74

Merged

countvajhula merged 441 commits into

mainfrom

lets-write-a-qi-compiler

Jan 12, 2024

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 3, 2024

add tests to check known counterexamples to seeming equivalences
countvajhula
committed
a63dc73
collapse singleton threading form
countvajhula
committed
c7d781e
Commit wip from today's meeting
countvajhula
committed
54ce045
wip from today's meeting (ignition!)
countvajhula
committed
be0d12b
implement filter-map using multiple values instead of allocated data
countvajhula
committed
d529721
add version from ben with partially evaluated match, for completeness
countvajhula
committed
f5c691c
add the hand-coded iteration (upper bound on performance)
countvajhula
committed
f6432ea
add `range-map-sum` benchmark (used in St-Amour's writeup)
countvajhula
committed
2be8e56
minimally incorporate stream fusion into the compiler
countvajhula
committed
aef7f88
restore restorative optimization that was accidentally dropped
countvajhula
committed
d06df52
test for the `filter-map` functional pipeline
countvajhula
committed
931585e
use `racket -y` in running all tests to recompile if needed
countvajhula
committed
f3d5550
fix stream fusion (from today's qi meeting)
countvajhula
committed
d215fea
tidy - remove deforestation wip from benchmarking module
countvajhula
committed
b398594
WIP from today's meeting
countvajhula
committed
0970f7c
use the much-anticipated syntax-spec-v1 from the package index :)
countvajhula
committed
8039a5c
a couple more nonlocal testcases, using folds
countvajhula
committed
b59139d
add starter tests to validate compiler rewrite rules
countvajhula
committed
679de88
benchmark for functional pipeline using foldr
countvajhula
committed
b2b620f
foldr fusable stream terminator
countvajhula
committed
6ae27fd
WIP from today's meeting
countvajhula
committed
9ea1c97
remove old unused deforestation code
countvajhula
committed
cca3691
Implement `foldl` as a stream
countvajhula
committed
3b22288
fix compiler tests by adding chirality
countvajhula
committed
c4aff58
consistent naming in rewrite rules
countvajhula
committed
cc3c666
add a benchmark using `foldl`
countvajhula
committed
c875f60
rename stream-related syntax classes for clarity
countvajhula
committed
7a7af5d
Use macro-debugger-emit to see the process-bindings expansion step in macro stepper.

dzoep
authored and
countvajhula
committed
0c1794f
Make fix procedure compatible with syntax-parse rules following the find-and-map/qi specification.

dzoep
authored and
countvajhula
committed
624fa3b
macros for emitting expansion events for the macro stepper
countvajhula
committed
b527858
Fix nonlocal tests (use right threading) for deforestation optimizations, add preliminary support for multiple values in deforestation implementation.

dzoep
authored and
countvajhula
committed
c89a3a3
Revert "Fix nonlocal tests (use right threading) for deforestation optimizations, add preliminary support for multiple values in deforestation implementation."

dzoep
authored and
countvajhula
committed
317ba4b
Fix the tests again.

dzoep
authored and
countvajhula
committed
ab19e1e
Remove redundant inlining hint.

dzoep
authored and
countvajhula
committed
c13bc24
Preliminary fusion for no-argument range, supporting 1 or 2 values.

dzoep
authored and
countvajhula
committed
4e8f82d
Support for multiple arguments to curry in producer syntax class.

dzoep
authored and
countvajhula
committed
1af613a
Full support for range fusion.

dzoep
authored and
countvajhula
committed
ae1b744
Finish stream fusion for producers, transformers, consumers and all their combinations.

dzoep
authored and
countvajhula
committed
db9ae68
Move the current deforestation (both syntax and implementation parts) into a separate module.

dzoep
authored and
countvajhula
committed
11ee6f8
Fix missing requires (for ~literal matching), streamline procedures naming in implementation, move inline-compose1 too.

dzoep
authored and
countvajhula
committed
196096a
Cleanup unused pattern variable, implement car deforestation.

dzoep
authored and
countvajhula
committed
3011919
Re-add begin-encourage-inline based on the benchmarks.

dzoep
authored and
countvajhula
committed
71a7a94
add `range-map` benchmark
countvajhula
committed
8bbc6a7
fix compiler tests
countvajhula
committed
539fef6
add a (failing) test for deforestation in nested positions
countvajhula
committed
388b466
remove testing-related nesting in qi deforestation benchmark
countvajhula
committed
77e54df
validate that `range` deforestation doesn't harm performance
countvajhula
committed
9b0bb41
fix (most) compiler tests again
countvajhula
committed
ec912d4
remove outdated compiler rewrite rule
countvajhula
committed
e367192
normalization rule to collapse `values` inside a threading form
countvajhula
committed
af597f1
Add initial tests for the normalization pass
countvajhula
committed
a8ef2db
improve `range-map-sum` benchmark
countvajhula
committed
6e9d7c1
add a "long functional pipeline" benchmark
countvajhula
committed
4ff81d9
remove unused code
countvajhula
committed
0166ade
Replace `range-map` benchmark with `range-map-car`
countvajhula
committed
c3ead6f
failing unit test for range-map-car
countvajhula
committed
b282410
Partial implementation of producer prepare contract.

dzoep
authored and
countvajhula
committed
a06de73
Refactor normalize pass into its own module
countvajhula
committed
523004e
note a todo for normalization, and remove an outdated comment
countvajhula
committed
d03d30e
Fix car stream consumer, generate all possible contracts for range variants.

dzoep
authored and
countvajhula
committed
4bc3b0d
Deforestation producers - currying prepare in the right order, contract only for the resulting lambda expression.

dzoep
authored and
countvajhula
committed
7831720
Preliminary implementation of consumer contracts with car as an example.

dzoep
authored and
countvajhula
committed
5d5f798
Unified range producer syntax class + currying contracted pipeline.

dzoep
authored and
countvajhula
committed
e82fdb1
fusable-stream-producer: limit the number of arguments to range to 1 to 3

dzoep
authored and
countvajhula
committed
5f4b382
move a simplification from the code generation step to normalization
countvajhula
committed
0a05ecc
Simplify deforestation tests to high level assertions
countvajhula
committed
c6236a1
Handle a simplified host expression in the compiler
countvajhula
committed
f8eb731
failing compiler tests for deforesting templates
countvajhula
committed
ab28b13
Rewrite partial application to a use of a blanket template
countvajhula
committed
4f2331a
A test to validate pre-supplying keyword arguments
countvajhula
committed
302b8f3
remove unused import
countvajhula
committed
b8ec517
match blanket templates in transformers and consumers
countvajhula
committed
a6281ca
add a failing test to show an issue with bindings and currying
countvajhula
committed
99c11e6
Add support for #%fine-template in deforested consumers.

dzoep
authored and
countvajhula
committed
a4cad12
Add missing literal.

dzoep
authored and
countvajhula
committed
f05b068
Matching _ as ~datum.

dzoep
authored and
countvajhula
committed
7a6b10e
deforestation error reporting: implement partial de-expander for flows

dzoep
authored and
countvajhula
committed
3bd39cd
Currying for #%fine-template deforested producers.

dzoep
authored and
countvajhula
committed
5801a0b
Work on simplifying deforestation pass.

dzoep
authored and
countvajhula
committed
7e3101b
deforestation: simplify consumer syntax patterns

dzoep
authored and
countvajhula
committed
cbcc972
add a few compiler tests for deforestation
countvajhula
committed
bcb9c8a
deforestation: fix blanket template with all arguments, expand transformer patterns

dzoep
authored and
countvajhula
committed
6285a06
reorganize compiler tests along producer, transformer, etc.
countvajhula
committed
c3ce843
combinatorial deforestation tests for producers using templates
countvajhula
committed
687b8db
Simplify compiler tests
countvajhula
committed
b007d98
fix invalid test
countvajhula
committed
a123aba
Fix "anaphoric references" issue (resolves failing test)
countvajhula
committed
d1a522c
tests for keyword arguments in templates
countvajhula
committed
0bb126b
deforestation: fix blanket template currying

dzoep
authored and
countvajhula
committed
aca1a38
deforestation: report too many arguments for blanket templates in syntax phase

dzoep
authored and
countvajhula
committed
410cb59
deforestation: preliminary support for argument count limits in syntax phase

dzoep
authored and
countvajhula
committed
c1dacb4
deforestation: update prettify-flow-syntax de-expander to reflect latest syntax template changes

dzoep
authored and
countvajhula
committed
c0636ba
deforestation: unify range producer syntax patterns into one

dzoep
authored and
countvajhula
committed
354f337
deforestation: do not duplicate producer name for contracts and error messages

dzoep
authored and
countvajhula
committed
0787fc0
deforestation: unified producer curry maker

dzoep
authored and
countvajhula
committed
da22297
deforestation: fix new producer curry semantics for implicit list->cstream producer

dzoep
authored and
countvajhula
committed
052e5d7
deforestation: more patterns unification

dzoep
authored and
countvajhula
committed
d5fc793
deforestation: improve invalid argument count error messages for static arguments

dzoep
authored and
countvajhula
committed
4b3b93e
deforestation: fix error message

dzoep
authored and
countvajhula
committed
37d3643
deforestation: full blame information at runtime

dzoep
authored and
countvajhula
committed
f72a2aa
deforestation: add range producer semantic tests

dzoep
authored and
countvajhula
committed
eb54077
deforestation: fix support for #%fine-template in fusable-stream-transformer0 and add missing esc in range producer test

dzoep
authored and
countvajhula
committed
928fbb9
include the new producer tests in the deforestation test suite
countvajhula
committed
89f5cb7
add a couple more tests for kwargs in a blanket template
countvajhula
committed
f5071d5
reorganize compiler tests into a dedicated folder (collection)
countvajhula
committed
bea9ea0
avoid division by zero in comparing benchmark results
countvajhula
committed
74f5fe8
Make higher benchmark threshold the same ratio as lower one
countvajhula
committed
3be7a17
Normalization tests and improvements
countvajhula
committed
2852fbe
remove old comment which I think has been addressed
countvajhula
committed
16274d0
add a basic threading test
countvajhula
committed
95db922
remove outdated todo (again?)
countvajhula
committed
f582ceb
move general and debugging-related functions into their own modules
countvajhula
committed
7db8b7c
simplify definitions and imports across phases
countvajhula
committed
6d27154
fix literal `range` in tests
countvajhula
committed
80b08af
declare macro-debugger dependency
countvajhula
committed
e3a1433
use test-equal? in test-normalize
countvajhula
committed
e065803
unit tests for `find-and-map/qi`
countvajhula
committed
26292ab
organize high level compilation sequence more clearly
countvajhula
committed
5c539ab
"fix" bad test
countvajhula
committed
4215189
simplify some definitions
countvajhula
committed
fb9641b
Fix collapsing `values` in normalization
countvajhula
committed
b6e3ee0
clean up normalization rules by declaring datum literals
countvajhula
committed
849ea89
revert phase changes to deforest.rkt for now
countvajhula
committed
dc74148
Comment out failing test
countvajhula
committed
8d7d447
deforest: format some brackets

benknoble
authored and
countvajhula
committed
0b73678
fix tests not being run on `make test`
countvajhula
committed
897f55a
Add more counterexamples found by Ben
countvajhula
committed
013961d
counterexamples...
countvajhula
committed
4117389
don't duplicate left and right identity rule
countvajhula
committed
9c88685
remove unsound normalization rules
countvajhula
committed
39de7ca
remove failing compiler tests for the unsound rules
countvajhula
committed
d738621
define qi functions in a uniform way
countvajhula
committed
d2d0e53
fix "not currently expanding" issue
countvajhula
committed
acb5f25
Make error pattern in ~> more specific to avoid bad error message
countvajhula
committed
8e4db77
Remove invalid (yet passing on main) test
countvajhula
committed
4067182
reorder some tests
countvajhula
committed
a9d1dda
failing unit tests for desired binding behavior with `switch`
countvajhula
committed
862d9f5
add binding spec for `if` (inherited by `switch`)
countvajhula
committed
7bc1554
don't look for fixed point in deforestation as it's unnecessary
countvajhula
committed
03fb1b1
starter tests for the expander
countvajhula
committed
bc5702d
a few more tests for the expander
countvajhula
committed
94077a9
declare missing build dependency on syntax-spec
countvajhula
committed
4316909
clarify a comment
countvajhula
committed
aa141ff
Use de-expander to fix (in a temporary way) #134
countvajhula
committed
df43041
commit missing moved de-expander
countvajhula
committed
f390ced
expand de-expander to full core language (except feedback)
countvajhula
committed
2ed742c
Tests for more compile-time errors
countvajhula
committed
425f672
test to catch a syntax error
countvajhula
committed
4cd21b0
cover de-expander in tests
countvajhula
committed
226dcac
a comment
countvajhula
committed
0b3a9ff
some refactoring and tests for coverage
countvajhula
committed
a376715
remove unused arity default in `loom-compose`
countvajhula
committed
90c4e2e
removed unused functions from core (these are in extended/impl now)
countvajhula
committed
b759c20
more tests for coverage
countvajhula
committed
940f7e0
whoops, call the right function for deforestation
countvajhula
committed
c52be32
Add a test to validate that deforestation is applied anywhere
countvajhula
committed
06c2abf
comment out test since it doesn't pass (why?)
countvajhula
committed
9056cc8
Fix numeric `fanout` not getting optimized implementation
countvajhula
committed
38c36d7
Remove unused literal parser from the compiler
countvajhula
committed
f4bc60d
more test coverage
countvajhula
committed
976d9a5
uncommenting test since it seems like a legitimate failure
countvajhula
committed
a6b251f
fix test to use `deforest-pass` (still correctly failing)
countvajhula
committed
57d99d2
fix deforesting of nested positions
countvajhula
committed
dc7cc53
convert compile time error in a test instead of using `eval`
countvajhula
committed
7de1212
make `make test` much faster by excluding the qi-doc package
countvajhula
committed
af6d056
add tests to reveal premature termination of normalization
countvajhula
committed
2fcd15b
tests to check deforestation is applied in nested and independent positions
countvajhula
committed
57012c6
Attempt to fix mutual inconsistency between `fix` and `find-and-map`
countvajhula
committed
7f374e1
comment out a mysteriously failing test...
countvajhula
committed
b9ee296
More tests to cover deforestation

dzoep
authored and
countvajhula
committed
b688d79
Add an optimization "rules" test for the "weird bug"
countvajhula
committed
8dd5d11
Add a failing test to reveal another case we should optimize
countvajhula
committed
d299370
bind `introduce-qi-syntax` once and use it everywhere (CR)
countvajhula
committed
8a0ef7c
Use a private submodule for `reference-qi`
countvajhula
committed
4d51e37
fix jumbling of clauses in de-expander for `partition` and `try`
countvajhula
committed
b62fc47
Fix(?) weird syntax pair bug
countvajhula
committed
f9e6f3f
recipe for tagging syntax for testing using a template metafunction
countvajhula
committed
84337eb
Commit improved "weird syntax pair bug" fix from the meeting
countvajhula
committed
d93a561
First attempt at fixing the tests with the new syntax property
countvajhula
committed
eb7bb69
Fix failing compiler utils tests
countvajhula
committed
5b81d3f
Fix remaining compiler tests by attaching `nonterminal` syntax property
countvajhula
committed
4c962e2
improve test doc
countvajhula
committed
323adf9
improve a macro used in testing for comparing syntax
countvajhula
committed
f6d1dd7
Organize utilities related to the `nonterminal` property
countvajhula
committed
b4b2007
Expose the ability to invoke the expander on demand
countvajhula
committed
d6f4e0b
Failing test to reveal bug with multiple passes
countvajhula
committed
6fd05b5
propagate srcloc and props in tagging with syntax property
countvajhula
committed
38adbd9
rename a `util` module to (compiler) `pass`
countvajhula
committed
6e616d6
simplify expander tests by defining a `test-expand` macro
countvajhula
committed
8ba7aad
add an expander test and some comments
countvajhula
committed
0207d90
provide a "getter" for the `nonterminal` syntax property
countvajhula
committed
7c3a1b4
Simplify and fix compiler rules tests with macros
countvajhula
committed
507828a
simplify deforestation rules tests further with macros
countvajhula
committed
9986578
Use on-demand expansion in normalization tests
countvajhula
committed
2122a90
Fix normalization bugs revealed by the new tests!
countvajhula
committed
dd11d60
(Redundantly) tag expansion output with the nonterminal property
countvajhula
committed
6122096
Uncomment a test that is now mysteriously working ¯\_(ツ)_/¯
countvajhula
committed
b9ee7e0
refile another `nonterminal` property related utility
countvajhula
committed
1ffc504
move normalization and deforestation rules tests into dedicated modules
countvajhula
committed
649d233
restore one test to a (legimately) failing state
countvajhula
committed
f2341dd
remove unused test suite
countvajhula
committed
212406c
Fix "full cycle" test so it runs in phase 1
countvajhula
committed
c1dee81
Propagage `nonterminal` property after normalization
countvajhula
committed
6f2a668
Move remaining compiler rules tests into phase 1
countvajhula
committed
fbb0740
remove Racket 8.5 from CI matrix for testing purposes...
countvajhula
committed
687069a
bump racket test matrix version to 8.6
countvajhula
committed
b5add8e
bump racket version to 8.7?
countvajhula
committed
a71be35
bump to 8.10
countvajhula
committed
8203828
actually bump to 8.10..
countvajhula
committed
0c8a02a
try racket 8.9
countvajhula
committed
e100735
Adjust test matrix to reflect known compatibility
countvajhula
committed
305ea0d
only attach the nonterminal property to the toplevel expression
countvajhula
committed
578d6c4
update a comment
countvajhula
committed
9090dc4
try BC 8.5 again just in case
countvajhula
committed
a2fe501
revert to version 8.9 in the test matrix
countvajhula
committed
0d27f0f
update some comments
countvajhula
committed
ec73d9e
add 8.5 back; modify test workflow to run all jobs even if some fail
countvajhula
committed
2744c68
fix CI workflow so it wouldn't always show success
countvajhula
committed
59fedaa

Commits on Jan 5, 2024

try quoting srcloc structure
countvajhula
committed
73e1fa0
restore test workflow config
countvajhula
committed
c09b926
fix source location marshalling by converting to vector
countvajhula
committed
6c066c0

Commits on Jan 6, 2024

Commits on Jan 11, 2024

doc: update grammar for the flow macro
countvajhula
committed
ff84ac4
update advice on currying and partial application quirks
countvajhula
committed
5ac92d3
minor capitalization in a comment
countvajhula
committed
3dbd5d3
basic doc for the `as` binding form
countvajhula
committed
568d336
doc: counting flows
countvajhula
committed
5118880
mention performance gains in some parts of the docs
countvajhula
committed
cc35173
add a placeholder doc page for the compiler (and expander)
countvajhula
committed
16ddc9a
document an error re: ambiguous binding
countvajhula
committed
36e9896
fix tab indent
countvajhula
committed
e4ddfe7
fix a typo
countvajhula
committed
e52aeda
remove unused test dependency on syntax-spec
countvajhula
committed
9b2e6a3
doc: avoid empty production of `literal`
countvajhula
committed
009fa0e
doc: basic example of binding more than one value with `as`
countvajhula
committed
30fdffe
doc: variable scoping rules
countvajhula
committed
87758ad
doc: minor improvements
countvajhula
committed
e425d93

Commits on Jan 12, 2024