Skip to content

Commit bd65d46

Browse files
authored
Merge pull request #49 from okybaca/improvements
Improvements
2 parents e95debf + a5ee183 commit bd65d46

File tree

8 files changed

+259
-35
lines changed

8 files changed

+259
-35
lines changed

docs/contribute.md

Lines changed: 81 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,105 @@
1-
# How to contribute to source code and documentation?
1+
# How to contribute to YaCy
2+
3+
4+
## Be active in the forum
5+
6+
[Community forum](https://community.searchlab.eu) keeps the community alive.
7+
If you're advanced user, good place to start is helping the others in
8+
the forum and sharing your knowledge there.
9+
10+
11+
## Report a bug
12+
13+
If you find a bug or you want to suggest an improvement, fill the
14+
[github issue](https://github.com/yacy/yacy_search_server/issues).
15+
16+
Please use factual and technical language and try to describe the bug in
17+
details.
18+
19+
Focus on what behavior you did expect and what YaCy did instead of it.
20+
21+
See [the log](operation/logging.md) and try to find and attach the
22+
appropriate log lines. Increase the log
23+
[verbosity](operation/logging.md#verbosity), if neccessary.
24+
25+
Be sure, that the issue could be replicated (describe what to do to see
26+
the malfunction again). Problem happening just once, without the possiblity
27+
to make it happen again, could be hardly fixed.
28+
29+
Pace of issue-fixing is currently quite low, so if you can, repair the
30+
function yourself and do a github pull-request (see below).
31+
32+
33+
## Help writting documentation
34+
35+
YaCy has many more functions than described in the [documentation](docs.md)
36+
and is still heavily under-documented. You can help others by writting about your
37+
favourite feature, updating the old pages, fixing installation guide for
38+
your platform, correcting the mistakes or just spell-checking. See
39+
[github repository of documentation](https://github.com/yacy/yacy_net_homepage),
40+
[documentation issues](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3ADocumentation)
41+
and github guide bellow.
42+
43+
44+
## Help developing YaCy
45+
46+
If you're a Java wizard, you're most warmly welcomed to be part of the
47+
development!
48+
49+
Although YaCy is developed [since 2003](https://en.wikipedia.org/wiki/YaCy), and still maintained,
50+
progress is sometimes slow. Therefore chances for including your code to
51+
mainstream are very high.
52+
53+
You can pick an [issue](https://github.com/yacy/yacy_search_server/issues) to solve.
54+
They're well tagged with [labels](https://github.com/yacy/yacy_search_server/labels) such as:
55+
[good first issue](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3A%22good%20first%20issue%22),
56+
[bug](https://github.com/yacy/yacy_search_server/issues?q=is%3Aissue%20state%3Aopen%20label%3Abug),
57+
[crawler](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3Acrawler),
58+
[search](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3Asearch),
59+
[index](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3Aindex),
60+
[network](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3Anetwork),
61+
[releasing](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3Areleasing),
62+
[developer](https://github.com/yacy/yacy_search_server/issues?q=state%3Aopen%20label%3A%22developer%20issue%22),
63+
etc.
64+
65+
You can improve what annoys you, personally, or craft a feature you like.
66+
67+
Before adding a major feature, consult
68+
[@orbiter](https://github.com/Orbiter), the main developer, or
69+
[the forum](https://community.searchlab.eu/).
70+
71+
72+
73+
## Step by step guide for Github
274

375
Basically, your contribution to the code and documentation is possible using
476
github.com. Create account there, fork the official repository, clone it to
577
your local machine, make a branch, modify files, commit changes to github.
678
Finally make a pull request, so your contribution could be merged into
779
master branch.
880

9-
Step by step guide:
1081

11-
(or follow the github contribution guide
12-
https://docs.github.com/en/get-started/quickstart/contributing-to-projects)
82+
(or follow the [github contribution guide](https://docs.github.com/en/get-started/quickstart/contributing-to-projects))
1383

1484

1585

16-
## this is for the first time only
86+
### this is for the first time only
1787

1888
* create a github.com account
1989

2090
* log into github
2191

22-
* fork repository https://github.com/yacy/yacy_search_server for a yacy
92+
* fork repository <https://github.com/yacy/yacy_search_server> for a yacy
2393
software modification,
24-
or https://github.com/yacy/yacy_net_homepage for documentation editing,
94+
or <https://github.com/yacy/yacy_net_homepage> for documentation editing,
2595
respectively (use a 'fork' button on top right part of github.com)
2696

2797
* your own fork is now at url:
2898
https://github.com/YOURUSERNAMEHERE/yacy_net_homepage
2999
or:
30100
https://github.com/YOURUSERNAMEHERE/yacy_search_server
31101

32-
* set-up a ssh key using this guide:
33-
https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account
102+
* set-up a ssh key using [this guide](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account)
34103

35104
* on your local machine, clone your new fork using commandline:
36105
```
@@ -48,7 +117,7 @@ https://docs.github.com/en/get-started/quickstart/contributing-to-projects)
48117
git config --global user.name "YOURUSERNAMEHERE"
49118
```
50119

51-
## this is the regular workflow:
120+
### for every next change
52121

53122
* create a new branch under your working directory:
54123
```
@@ -77,9 +146,9 @@ https://docs.github.com/en/get-started/quickstart/contributing-to-projects)
77146
to the master yacy branch.
78147

79148
* you can see all open pull request by you and other people at:
80-
https://github.com/yacy/yacy_search_server/pulls
149+
<https://github.com/yacy/yacy_search_server/pulls>
81150
or
82-
https://github.com/yacy/yacy_net_homepage/pulls
151+
<https://github.com/yacy/yacy_net_homepage/pulls>
83152

84153
* wait until the mantainers merge your pull request
85154

docs/docs.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,11 @@
2626

2727

2828
## Operation
29-
* [Index Creation - Crawl Start](operation/crawlstart_p.md)
29+
* [Advanced Crawler](operation/crawlstart_p.md)
3030
* [Setting the ranking rules](operation/ranking.md)
3131
* [YaCy config settings](operation/yacy_conf.md)
3232
* [RWI index distribution](operation/rwi-index-distribution.md)
33+
* [Yacy Packs](operation/yacy-pack.md)
3334
* [Autoupdate](operation/autoupdate.md)
3435
* [Portforwarding](operation/portforwarding.md)
3536
* [Using the YaCy Front-End over HTTPS](operation/yacyoverhttps.md)
@@ -52,7 +53,8 @@
5253
## Converted from old-wiki
5354
may be outdated, you can help the community by checking and [improving](contribute.md) the pages
5455

55-
...
56+
* [Index Export and Import](operation/index-export-import.md)
57+
* [Advanced Search Parameters](operation/search-parameters.md)
5658

5759
## Old and obsolete
5860
The original YaCy wiki is closed now (no new registration or editing) and

docs/download_installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ If you don't have Docker installed, get it from [https://docs.docker.com/get-doc
2323

2424

2525
* <a class="btn btn-success btn" href="https://download.yacy.net/yacy_v1.924_20201214_10042.exe" role="button">Download YaCy for Windows</a> from [https://download.yacy.net/yacy_v1.924_20201214_10042.exe](https://download.yacy.net/yacy_v1.924_20201214_10042.exe)
26-
* <a class="btn btn-success btn" href="https://download.yacy.net/yacy_v1.930_202405130205_59c0cb0f3.tar.gz" role="button">Download Yacy for Linux</a> from [https://download.yacy.net/yacy_v1.930_202405130205_59c0cb0f3.tar.gz](https://download.yacy.net/yacy_v1.930_202405130205_59c0cb0f3.tar.gz)
26+
* <a class="btn btn-success btn" href="https://download.yacy.net/yacy_v1.940_202405270005_70454654f.tar.gz" role="button">Download Yacy for Linux</a> from [https://download.yacy.net/yacy_v1.940_202405270005_70454654f.tar.gz](https://download.yacy.net/yacy_v1.940_202405270005_70454654f.tar.gz)
2727
* <a class="btn btn-success btn" href="https://download.yacy.net/yacy_v1.930_202405130205_59c0cb0f3.dmg" role="button">Download YaCy for macOS</a> from [https://download.yacy.net/yacy_v1.930_202405130205_59c0cb0f3.dmg](https://download.yacy.net/yacy_v1.930_202405130205_59c0cb0f3.dmg)
2828
* <a class="btn btn-success btn" href="https://release.yacy.net/yacy_latest.tar.gz" role="button">Download latest developer release</A> for Linux from [https://release.yacy.net/](https://release.yacy.net/)
2929

docs/img/indexexpimp1.jpg

182 KB
Loading

docs/img/indexexpimp2.jpg

185 KB
Loading
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Index export and import
2+
3+
Since the development version 1.83 build 9250 YaCy has now the long
4+
awaited feature to handle the index data in a more convenient way: An
5+
ex- and import feature has been implemented.
6+
7+
It is no longer necessary to study complicated manuals for the merge of
8+
two solr indexes and/or setup an additional stand-alone instance for
9+
index merging tasks - thanks to the great work of Orbiter - YaCy
10+
delivers now a powerful ex- and import feature out-of-the-box\!
11+
12+
## How to do that?
13+
14+
Here's a short tutorial:
15+
16+
![Index Export screenshot](../img/indexexpimp1.jpg)]
17+
18+
1\. On the machine you want to export the index data open a browser and
19+
navigate to <http://localhost:8090/IndexExport_p.html>
20+
21+
Leave the settings as is, XML (Rich and full-text Solr data, one
22+
document per line in one large xml file, can be processed with shell
23+
tools, can be imported with `DATA/SURROGATE/in/`) because it's the best
24+
choice for the consistency of your data.
25+
26+
2\. Press the 'Export URLs'-Button and grab some coffee :-)
27+
28+
![Index Export screenshot](../img/indexexpimp2.jpg)
29+
30+
3\. On the machine you want to import the index data, simply put the
31+
exported XML file into the following subdirectory of YaCy: `\DATA\SURROGATES\in`
32+
33+
You can do this during YaCy is running - No need to shut it down first\!
34+
35+
4\. Voilà - The import process starts automatically and is blazing fast
36+
- even on older machines. After the import process is completed, you can
37+
search trough the data instantly - reindexing is superfluous.
38+
39+
40+
41+
42+
43+
_Converted from
44+
<http://wiki.yacy.de/index.php?title=En:IndexExpImp&oldid=22656>“, may be
45+
outdated_
46+
47+
48+
49+
Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Using Advanced Search Parameters
2+
3+
Most search requests usually contain just one word. If more than one
4+
word is used, all words will be assumed to be part of an AND relation.
5+
Besides searching for certain words with YaCy, you can use more advanced
6+
methods to put a search request:
7+
8+
## Excluding a Word
9+
10+
To exclude a word from a search, a minus (`-`) can be used: for example,
11+
if searching for *jaguar* produces too many results associated with cars
12+
when in fact you are looking for the animal, searching for `jaguar -car`
13+
might lead to better results.
14+
15+
## NEAR
16+
17+
`NEAR` can be used to rank results higher if search words appear in the
18+
text close to each other. Example: `apache server NEAR`.
19+
20+
It does not matter where NEAR is located in the search term. `apache
21+
server NEAR` and `apache NEAR server` should return the same results.
22+
23+
## site:
24+
25+
`findsomething site:yacy.net` will limit the results to the domain
26+
yacy.net, subdomains excluded. See `tld:` operator also.
27+
28+
## tld:
29+
30+
`findsomething tld:co.uk` will limit the results to domains ending with
31+
`*.co.uk.*` This can also be used to search on subdomains.
32+
33+
## inurl:
34+
35+
`findsomething inurl:source` will limit the results to URLs which
36+
contain the phrase "source".
37+
38+
## filetype:
39+
40+
`findsomething filetype:pdf` will limit the results to URLs which end
41+
with `.pdf`.
42+
43+
## LANGUAGE:
44+
45+
`findsomething LANGUAGE:en` will rank results in English language
46+
higher. (Note: Language detection is still very experimental\!)
47+
48+
## RECENT
49+
50+
`findsomething RECENT` will rank recently crawled pages higher.
51+
52+
## Protocol
53+
54+
`findsomething /ftp` will limit the results to URLs with FTP protocol
55+
(ftp://). List of available protocol: `/https` `/http` `/ftp` `/smb` or
56+
`/file`
57+
58+
## author:
59+
60+
`findsomething author:busch` will limit the results to URLs with author
61+
"busch". `findsomething author:(Wilhelm busch)` will limit the results
62+
to URLs with author "Wilhelm busch".
63+
64+
## Date Search
65+
66+
To search for results mentioning a specific date.
67+
68+
*Note: to support date search [solr index field](../dev/solr-schema.md#optional-but-recommended) `dates_in_content_dts` must be
69+
switched on*
70+
71+
72+
### on:
73+
74+
`findsomething on:2016/01/01` will limit the results to URLs which
75+
contain the given date in the content.
76+
77+
78+
### from: to:
79+
80+
`findsomething from:2016/01/01` will limit the results to URLs which
81+
contain a date on or after the `from:` parameter.
82+
83+
`findsomething to:2016/01/01` will limit the results to URLs which
84+
contain a date on or before the `to:` parameter.
85+
86+
Both can be combinded to limit results to the given date range
87+
`findsomething from:2016/01/01 to:2016/12/31`
88+
89+
90+
91+
92+
93+
_Converted from
94+
<https://wiki.yacy.net/index.php?title=En:SearchParameters&oldid=22861> may
95+
be outdated_
96+
97+
98+
99+

0 commit comments

Comments
 (0)