Merge pull request #173 Version 1.40

Prepare Version 1.40

Author: gnadelwartz

README.html

@@ -223,8 +223,8 @@ <h2><img align="middle" src="https://raw.githubusercontent.com/odb/official-bash
 <p>Released to the public domain wherever applicable. Elsewhere, consider it released under the <a href="http://www.wtfpl.net/txt/copying/">WTFPLv2</a>.</p>
 <p>Linted by <a href="https://github.com/koalaman/shellcheck">#ShellCheck</a></p>
 <h2>Prerequisites</h2>
-<p>Uses <a href="http://github.com/dominictarr/JSON.sh">JSON.sh</a> and the magic of sed.</p>
-<p>Bashbot is written in bash. It depends on commands typically available in a Linux/Unix Environment. For more concrete information on the common commands provided by recent versions of <a href="https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands">coreutils</a>, <a href="https://en.wikipedia.org/wiki/BusyBox#Commands">busybox</a> or <a href="https://landley.net/toybox/help.html">toybox</a>, see <a href="doc/7_develop.md#common-commands">Developer Notes</a>.</p>
+<p>Uses <a href="http://github.com/dominictarr/JSON.sh">JSON.sh</a>/<a href="https://github.com/step-/JSON.awk">JSON.awk</a> and the magic of sed.</p>
+<p>Bashbot is written in bash. It depends on commands typically available in a Linux/Unix Environment. For more information on commands provided by recent versions of <a href="https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands">coreutils</a>, <a href="https://en.wikipedia.org/wiki/BusyBox#Commands">busybox</a> or <a href="https://landley.net/toybox/help.html">toybox</a>, see <a href="doc/7_develop.md#common-commands">Developer Notes</a>.</p>
 <p><strong>Note for MacOS and BSD Users:</strong> Bashbot will not run without installing additional software as it uses modern bash and (gnu) grep/sed features. See <a href="doc/0_install.md">Install Bashbot</a>.</p>
 <p><strong>Note for embedded systems:</strong> You need to install a "real" bash as the vanilla installation of busybox or toybox is not sufficient. See <a href="doc/0_install.md">Install Bashbot</a>.</p>
 <p>Bashbot <a href="https://github.com/topkecleon/telegram-bot-bash">Documentation</a> and <a href="https://github.com/topkecleon/telegram-bot-bash/releases">Downloads</a> are available on <a href="https://www.github.com">www.github.com</a>.</p>
@@ -284,7 +284,8 @@ <h2>Documentation</h2>
 <li>Setup your environment</li>
 <li>Bashbot test suite</li>
 </ul></li>
-<li><a href="examples/README.md">Examples Directory</a></li>
+<li><a href="examples">Examples Directory</a></li>
+<li><a href="examples/webhook">Webhook Example</a></li>
 </ul>
 <h3>Your very first bashbot in a nutshell</h3>
 <p>To install and run bashbot you need access to a Linux/Unix command line with bash, a <a href="https://telegram.org">Telegram client</a> and a mobile phone <a href="https://telegramguide.com/create-a-telegram-account/">with a Telegram account</a>.</p>
@@ -340,7 +341,7 @@ <h2>Security Considerations</h2>
 <p>Running a Telegram Bot means it is connected to the public and you never know what's send to your Bot.</p>
 <p>Bash scripts in general are not designed to be bulletproof, so consider this Bot as a proof of concept. Bash programmers often struggle with 'quoting hell' and globbing, see <a href="https://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells">Implications of wrong quoting</a>.</p>
 <p>Whenever you are processing input from untrusted sources (messages, files, network) you must be as careful as possible (e.g. set IFS appropriately, disable globbing with <code>set -f</code> and quote everything). In addition remove unused scripts and examples from your Bot (e.g. everything in <code>example/</code>) and disable/remove all unused bot commands.</p>
-<p>It's important to escape or remove <code>$</code> in input from user, files or network (<em>as bashbot does</em>). One of the powerful features of Unix shells is variable and command substitution using <code>${}</code> and<code>$()</code> can lead to remote code execution (RCE) or remote information disclosure (RID) bugs if unescaped <code>$</code> is included in untrusted input (e.g. <code>$$</code> or <code>$(rm -rf /*)</code>).</p>
+<p>It's important to escape or remove <code>$</code> in input from user, files or network (<em>as bashbot does</em>). One of the powerful features of Unix shells is variable and command substitution using <code>${}</code> and <code>$()</code> can lead to remote code execution (RCE) or remote information disclosure (RID) bugs if unescaped <code>$</code> is included in untrusted input (e.g. <code>$$</code> or <code>$(rm -rf /*)</code>).</p>
 <p>A powerful tool to improve your scripts is <code>shellcheck</code>. You can <a href="https://www.shellcheck.net/">use it online</a> or <a href="https://github.com/koalaman/shellcheck#installing">install shellcheck locally</a>. Shellcheck is used extensively in bashbot development to ensure a high code quality (e.g. it's not allowed to push changes without passing all shellcheck tests). In addition bashbot has a <a href="doc/7_develop.md">test suite</a> to check if important functionality is working as expected.</p>
 <h3>Use printf whenever possible</h3>
 <p>If you're writing a script that accepts external input (e.g. from the user as arguments or the file system), you shouldn't use echo to display it. <a href="https://unix.stackexchange.com/a/6581">Use printf whenever possible</a>.</p>
@@ -349,8 +350,9 @@ <h3>Run your Bot as a restricted user</h3>
 <p><strong>Never run your Bot as root, this is the most dangerous you can do!</strong> Usually the user 'nobody' has almost no rights on Linux/Unix systems. See <a href="doc/4_expert.md">Expert use</a> on how to run your Bot as an other user.</p>
 <h3>Secure your Bot installation</h3>
 <p><strong>Your Bot configuration must not be readable by other users.</strong> Everyone who can read your Bots token is able to act as your Bot and has access to all chats the Bot is in!</p>
-<p>Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in<code>config.jssh</code> must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to<code>count.jssh</code> and <code>data-bot-bash</code> only, all other files must be write protected.</p>
-<p>To set access rights for your bashbot installation to a reasonable default run<code>sudo ./bashbot.sh init</code> after every update or change to your installation directory.</p>
+<p>Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in <code>config.jssh</code> must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to <code>count.jssh</code>, <code>data-bot-bash/</code> and <code>logs/</code> only, all other files must be write protected.</p>
+<p>To set access rights for your bashbot installation to a reasonable default run <code>sudo ./bashbot.sh init</code> after every update or change to your installation directory.</p>
+<p><em>Note</em>: Keep old log files in a safe place or even better delete them, they are GDPR relevant and <a href="https://github.com/topkecleon/telegram-bot-bash/issues/174">may contain information</a> you don't want to be public.</p>
 <h2>FAQ</h2>
 <h3>Is this Bot insecure?</h3>
 <p>Bashbot is not more (in)secure than a Bot written in another language. We have done our best to make it as secure as possible. But YOU are responsible for the bot commands you wrote and you should know about the risks ...</p>
@@ -364,8 +366,8 @@ <h3>Why Bash and not the much better xyz?</h3>
 <li>no database, not event driven, not object oriented ...</li>
 </ul>
 <h3>Can I have the single bashbot.sh file back?</h3>
-<p>At the beginning bashbot was simply the file<code>bashbot.sh</code> that you could copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.</p>
-<p>Hey no problem, if you are finished with your cool bot, run<code>dev/make-standalone.sh</code> to create a stripped down version of your bot containing only 'bashbot.sh' and 'commands.sh'! For more information see <a href="doc/7_develop.md">Create a stripped down version of your Bot</a>.</p>
+<p>At the beginning bashbot was simply the file <code>bashbot.sh</code> that you could copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.</p>
+<p>Hey no problem, if you are finished with your cool bot, run <code>dev/make-standalone.sh</code> to create a stripped down version of your bot containing only 'bashbot.sh' and 'commands.sh'! For more information see <a href="doc/7_develop.md">Create a stripped down version of your Bot</a>.</p>
 <h3>Can I send messages from CLI and scripts?</h3>
 <p>Of course you can send messages from command line and scripts! Simply install bashbot as <a href="#Your-really-first-bashbot-in-a-nutshell">described here</a>, send the message '/start' to set yourself as botadmin and then stop the bot with <code>./bashbot.sh stop</code>.</p>
 <p>Bashbot provides some ready to use scripts for sending messages from command line in <code>bin/</code> dir, e.g. <code>send_message.sh</code>.</p>
@@ -390,6 +392,6 @@ <h3>Blocked by telegram?</h3>
 <p>@Gnadelwartz</p>
 <h2>That's it all guys!</h2>
 <p>If you feel that there's something missing or if you found a bug, feel free to submit a pull request!</p>
-<h4>$$VERSION$$ v1.30-0-g3266427</h4>
+<h4>$$VERSION$$ v1.40-dev-34-g1440d56</h4>
 </body>
 </html>

README.md

@@ -11,10 +11,10 @@ Elsewhere, consider it released under the [WTFPLv2](http://www.wtfpl.net/txt/cop
 Linted by [#ShellCheck](https://github.com/koalaman/shellcheck)
 
 ## Prerequisites
-Uses [JSON.sh](http://github.com/dominictarr/JSON.sh) and the magic of sed.
+Uses [JSON.sh](http://github.com/dominictarr/JSON.sh)/[JSON.awk](https://github.com/step-/JSON.awk) and the magic of sed.
 
 Bashbot is written in bash. It depends on commands typically available in a Linux/Unix Environment.
-For more concrete information on the common commands provided by recent versions of [coreutils](https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands), [busybox](https://en.wikipedia.org/wiki/BusyBox#Commands) or [toybox](https://landley.net/toybox/help.html), see [Developer Notes](doc/7_develop.md#common-commands).
+For more information on commands provided by recent versions of [coreutils](https://en.wikipedia.org/wiki/List_of_GNU_Core_Utilities_commands), [busybox](https://en.wikipedia.org/wiki/BusyBox#Commands) or [toybox](https://landley.net/toybox/help.html), see [Developer Notes](doc/7_develop.md#common-commands).
 
 **Note for MacOS and BSD Users:** Bashbot will not run without installing additional software as it uses modern bash and (gnu) grep/sed features. See [Install Bashbot](doc/0_install.md).
 
@@ -63,7 +63,8 @@ Bashbot [Documentation](https://github.com/topkecleon/telegram-bot-bash) and [Do
     * Modules, addons, events
     * Setup your environment
     * Bashbot test suite
-* [Examples Directory](examples/README.md)
+* [Examples Directory](examples)
+* [Webhook Example](examples/webhook)
 
 ### Your very first bashbot in a nutshell
 
@@ -146,7 +147,7 @@ Whenever you are processing input from untrusted sources (messages, files, netwo
 from your Bot (e.g. everything in `example/`) and disable/remove all unused bot commands.
 
 It's important to escape or remove `$` in input from user, files or network (_as bashbot does_).
-One of the powerful features of Unix shells is variable and command substitution using `${}` and`$()` can lead to remote code execution (RCE) or remote information disclosure (RID) bugs if unescaped `$` is included in untrusted input (e.g. `$$` or `$(rm -rf /*)`).
+One of the powerful features of Unix shells is variable and command substitution using `${}` and `$()` can lead to remote code execution (RCE) or remote information disclosure (RID) bugs if unescaped `$` is included in untrusted input (e.g. `$$` or `$(rm -rf /*)`).
 
 A powerful tool to improve your scripts is `shellcheck`. You can [use it online](https://www.shellcheck.net/) or
 [install shellcheck locally](https://github.com/koalaman/shellcheck#installing). Shellcheck is used extensively in bashbot development
@@ -168,9 +169,11 @@ For the same reason every file your Bot can read is in danger of being disclosed
 ### Secure your Bot installation
 **Your Bot configuration must not be readable by other users.** Everyone who can read your Bots token is able to act as your Bot and has access to all chats the Bot is in!
 
-Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in`config.jssh` must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to`count.jssh` and `data-bot-bash` only, all other files must be write protected.
+Everyone with read access to your Bot files can extract your Bots data. Especially your Bot config in `config.jssh` must be protected against other users. No one except you should have write access to the Bot files. The Bot should be restricted to have write access to `count.jssh`, `data-bot-bash/` and `logs/` only, all other files must be write protected.
 
-To set access rights for your bashbot installation to a reasonable default run`sudo ./bashbot.sh init` after every update or change to your installation directory.
+To set access rights for your bashbot installation to a reasonable default run `sudo ./bashbot.sh init` after every update or change to your installation directory.
+
+*Note*: Keep old log files in a safe place or even better delete them, they are GDPR relevant and [may contain information](https://github.com/topkecleon/telegram-bot-bash/issues/174) you don't want to be public.
 
 ## FAQ
 
@@ -188,9 +191,9 @@ Well, that's a damn good question... maybe because I'm a Unix admin from the sto
 - no database, not event driven, not object oriented ...
 
 ### Can I have the single bashbot.sh file back?
-At the beginning bashbot was simply the file`bashbot.sh` that you could copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.
+At the beginning bashbot was simply the file `bashbot.sh` that you could copy everywhere and run the bot. Now we have 'commands.sh', 'mycommands.sh', 'modules/*.sh' and much more.
 
-Hey no problem, if you are finished with your cool bot, run`dev/make-standalone.sh` to create a stripped down version of your bot containing only
+Hey no problem, if you are finished with your cool bot, run `dev/make-standalone.sh` to create a stripped down version of your bot containing only
 'bashbot.sh' and 'commands.sh'! For more information see [Create a stripped down version of your Bot](doc/7_develop.md).
 
 ### Can I send messages from CLI and scripts?
@@ -238,4 +241,4 @@ See `mycommnds.sh.dist` for an example.
 
 If you feel that there's something missing or if you found a bug, feel free to submit a pull request!
 
-#### $$VERSION$$ v1.30-0-g3266427
+#### $$VERSION$$ v1.40-0-gf9dab50