Let's keep Xray alive

[fdionisi]

As posted in the original repository letting Xray die would be a real pity.

I was hoping in a response from its maintainers – I still have questions about where Memo should fit into Xray or next steps towards JS extensions, but I also believe it's time to prove the real interest from the community if there is some.

I apologise in advance if the notification for this post was undesired for some tagged users! My purpose is to gather some enthusiastic developers to help me keep Xray alive and to get maintainers' "blessing".

The purpose of this fork is mainly to show real interest, and hopefully, it will be eventually merged back.

Below, I will explain the steps that, to me, seem a natural continuation of the project.

It should be needless to say that the "roadmap" must not be taken as a given and suggestion are more than welcome!

Planning

While they are focusing on better shaping Memo, I think a good starting point would be merging the outstanding pull-requests.

• Initial Appveyor Config atom-archive/xray#57 @chgibb
  Better to push this for later (until named pipes are implemented)

• Add mouse "click and drag" to select text atom-archive/xray#107 @pranaygp
  Covered in Add/implement mouse selections #3

• Add multi cursor click selections atom-archive/xray#108 @pranaygp

• Buffer Selection Clipping atom-archive/xray#132 @coler706
  Covered in Clip selection #5

• Fix panic at index out of bound atom-archive/xray#172 @hourann
  Covered in Hourann/fix panic at index out of bound #4

Tests should be added, where needed, to ensure future proof.

Would be great to have the above PRs redirected to here!

---

There are few simple TODOs around the codebase that could be implemented easily:

• Use requestIdleCallback as background executor

• Use Vec<u8> for outgoing messages in xray_core

• Reverse selection if end < start

More complex one that may need some background:

• Implement PeerList UI

• Move theme on the server

• Move key bindings on the server

Memo has some too. This may be sent to the main repo straight away:

• Only check for cycles in debug mode

• Only check for cycles if the child was moved and is a directory.

• Switch epoch convergence I

• Switch epoch convergence II

---

To better engage the whole community, Windows support could be helpful:

• Use named pipes for Windows OSs

---

Along those, fixes and improvements to headless feature will be needed.

As first, the browser version should work.

• Fix Xray Server encoder

  I spent quite some time investigating this issue – I'm not a Rust expert and also not super comfortable with low-level programming. Eventually, I was able to workaround this issue setting up an overly-big frame size.

  // ...
  let transport = codec::length_delimited::Builder::new()
      .max_frame_length(2 * 1024 * 1024 * 1024) // 😱
      .new_framed::<_, Bytes>(socket);
  // ...

  Still code on my local machine

  The code above solve handling connection to Xray repository, but it would be great to have some help from some Rustacean that knows how to implement a streamed representation for in-memory trees, as suggested in the linked issue comment.

It would be great to add the possibility to connect the Electron app through socket as well if it is not there yet (I may have missed that).

• Headless connection from Electron

---

The repository is still quite small and experimental, therefore breaking changes are not a problem and until is not a real product keeping dependencies up to date should be straight forward.

• Update JavaScript/WASM dependencies

  • xray_ui
  • xray_browser
  • xray_electron
  • xray_wasm

I personally think that TypeScript is a great tool to build solid production ready products. An effort will be made to move the above packages to rely on static typing, but not until the initial direction/rationale gets more clear.

The issue I found trying this out is the absence of typings for styletron libraries – at least, until v5 will be released. Since waiting for the new version to be released wouldn't be ideal, the possibilities are two:

• write typings along the way for used styletorn features, or
• move to a TypeScript supported CSS-in-Js library

Before even considering moving to a different library, I would love to understand the rationale
behind choosing styletorn in the first place.

Once the current version will be totally ported to TypeScript, React should be updated – Context API, memo, hooks...

• Context API

  • Re-design App Context
  • Re-design Action Context
  • Re-design Theme Context
  • Re-design View Context

• Hooks

Alongside JavaScript/TypeScript migration, would be great to update Rust crates. Also here, some help from the Rust community would be great.

• Update to latest dependencies
• Migrate to 2018 edition

---

And now more seirous stuff where help from maitainers would definitely be needed to implement the following properly:

• Implement syntax highlight (tree-sitter)
• Implement Memo into Xray
• Refactor and enable discussions

---

To make the editor a little bit usable, additional basic features are needed:

• Undo history
• Find and replace in a buffer
• Local project-wide search
• Remote project-wide search

---

With the above tasks completed, it should be possible to eat our own dog's food: use Xray to make Xray a hackable editor.

• Clarify theming

• JS extensions

  • spawn-server
  • Investigate Deno

I'm not sure if spawn-server was actually an initial experiment behind Xray's JS extensions, but after looking at @ry's talk about deno I started thinking it could be a good fit for this purpose.

---

/cc @nathansobo @as-cii @HotelCalifornia @fuunnx @sadovnychyi @b-fuze @mikemcbride @max-sixty @Avi-D-coder @deankevorkian @Pustelto

[nathansobo]

Honored that you care. A lot in the air for me right now both personally and professionally. I’d like some more time to reflect before commenting further.

[fdionisi]

I’d like some more time to reflect before commenting further.

Sure thing! Take your time and please don't feel pressure on this.

[deankevorkian]

Hi! I appreciate the effort you plan to put into this project. I hope that we as a community will be able to push ongoing efforts and perhaps start our own efforts.
I think most of the community that is interested in xrays success comes from the Javascript world and, just as you said, is not familiar with Rust.
Because of that I think focusing on merging existing pull requests and correcting semi broken behaviours would be a good start, but I think that major changes (technically and architecturally, be it the way processes communicate or be it moving from JS to TS) may be too big and might be rushed. We have to remember that this project is extremely experimental and had a lot of breaking changes. Because of its nature, we also don't know the reasons for all the choices Nathan and Antonio have made during the process. There might be a reason for why they didn't go with typescript, for example (because typescript already exists in atom's language server client).

My main point is - advancing and finishing open things are great for starters, but changing existing behaviours - especially since we aren't fully aware of why everything is made, and with Rust at all (and writing performant code with it - which is a somewhat painful point of Atom).
Perahps on the Javascript and WASM side it'll be safer to make changes that are somewhat breaking because, as a community, its an ecosystem we are more familiar and professional in.

(this was just my humble opinion).

Good luck with the effort, absolutely glad you started this and so quick after the news about xrays near future state, I'm not sure I'll be able to contribute much but I'll see where I can blend if at all. Thank you very much for this, and of course @nathansobo and @as-cii for the amazing work and research that thrilled the interest of the community!

[fdionisi]

Sure @deankevorkian – postponing huge changes until there's a better understanding behind all initial rationale make sense.

I still think that if somehow more and more people are going to contribute to this repo, the codebase should migrate to TypeScript – but that's an opinion, and I don't want start an argument on this now ☺️.

I'm not proficient with Rust as I am with JavaScript, but I look forward to getting better especially with a project that inspires me – I'm playing with it as much as possible during my free time. Of course, some input/directions would make my life easier but Rust docs and community are great, and I'm not afraid of diving in.

Thank you a lot for your suggestions – I will update my first comment on avoid making big changes/updates until there's a good understanding of why it was initially setup in this way.

[felixrabe]

@fdionisi For learning Rust, check out https://github.com/rust-lang/rustlings (can be a bit challenging depending how new you are) and https://exercism.io/tracks/rust (even easy problems make for good treasure hunting in std :) ).

[felixrabe]

@fdionisi And if you're up to it, consider posting to https://www.reddit.com/r/rust/ and/or https://users.rust-lang.org/ to get more eyes.

[leeola]

Since the lifespan of this project is in question, perhaps cutting features would be meaningful? Specifically, rather than planning on continuing support for xray-core, exploit the work being done on the Xi Editor and it's similarities to this project.

I'm not familiar with Xray or the differences between Xray/Xi, but as the discussion in #4 they do share many features.

As open source contributions tend to be short in supply, I think Xi should be heavily considered before planning on continuing support with xray-core. At the very least, we should have a good understanding of why we need xray-core, and why we can't make use of the considerable work being done over at Xi.

Super interested in @nathansobo 's thoughts on this now that the official project looks to be DoA.

[sadovnychyi]

I would agree with @leeola that trying to compete with xi without github support would be wasteful. They do share similar goals. Putting community work into memo alone seems much more realistic.

[fdionisi]

@leeola One of the main things in Xray would be the ability to have it running as a desktop app but also as an in-browser web app, therefore, being able to compile it down to WASM is really kind of mandatory.

I never focused too much on Xi, but after doing a search, it seems that they are not focused on that while Xray stands on that concept.

Besides that, collaboration first is another difference with Xi (please, correct me if I'm wrong).

[cztomsik]

I was about to suggest using https://github.com/cztomsik/stain for GUI part, it's an attempt to do something like react-native-desktop but using GPU renderer.

But browser support is something we will not have for a while. We still don't even support win10.

If you'd be interested, let me know, I'd like to do some text editor on top of that lib but I have zero experience with text editing problem domain.

[fdionisi]

@cztomsik I was looking at your work a few days ago hoping it would be possible to integrate it into Xray, but indeed, as said above being able to run in the browser it's quite important (I'm talking for myself here, I am not sure which was the direction intended by primary maintainers).

If you want to contribute anyway, I feel that your knowledge could be beneficial, especially for this ticket.

Thanks for reaching out! Hope your project continues well 😊.

[cztomsik]

I'm afraid there's not much I could add to that ticket. We are using pango, and because we need to compute line-height which is different from spacing, we need to hack it a bit so I'm not even sure how it would work with non-latin texts.

[dlight]

Hello, is there plans for rebranding? If this project is going forward, it's better to do a hardfork and adopt a new name. (unless Github makes https://github.com/atom-archive/xray a redirect to https://github.com/fdionisi/xray)