Skip to content

0.8.2 mapr 1912 r3 #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 494 commits into
base: master
Choose a base branch
from
Open

0.8.2 mapr 1912 r3 #1

wants to merge 494 commits into from

Conversation

ajgoade
Copy link

@ajgoade ajgoade commented Jun 10, 2021

What is this PR for?

A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html

What type of PR is it?

[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]

Todos

  • - Task

What is the Jira issue?

How should this be tested?

Outline the steps to test the PR here.

Screenshots (if appropriate)

Questions:

  • Does the licenses files need update?
  • Is there breaking changes for older versions?
  • Does this needs documentation?

zjffdu and others added 30 commits March 20, 2018 14:09
… for PySparkInterpreter'

### What is this PR for?
Trivial PR for removing the unnecessary message

### What type of PR is it?
[Improvement]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3290

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2882 from zjffdu/ZEPPELIN-3290 and squashes the following commits:

7bf8d6e [Jeff Zhang] ZEPPELIN-3290. Unnecessary message 'IPython is available, use IPython for PySparkInterpreter'

(cherry picked from commit 67582b8)
Signed-off-by: Jeff Zhang <[email protected]>
### What is this PR for?
Straightforward fix for livy security issue.

### What type of PR is it?
[Bug Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3357

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2887 from zjffdu/ZEPPELIN-3357 and squashes the following commits:

2fbbbeb [Jeff Zhang] ZEPPELIN-3357. Livy security mode is broken

(cherry picked from commit db716c8)
Signed-off-by: Jeff Zhang <[email protected]>
In current implementation some of the unescaped HTML get passed to frontend via BootstrapDialog, this PR is to escape those string (and sanitize the output).

[Improvement]

* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?

Author: Prabhjyot Singh <[email protected]>

Closes apache#2888 from prabhjyotsingh/applyEscapeBootstrapDialog and squashes the following commits:

757cfff [Prabhjyot Singh] apply _.Escape to BootstrapDialog

Change-Id: Icabd5e5713591929cb4ff9a41036f06ca99b6db8
(cherry picked from commit 645037b)
Signed-off-by: Prabhjyot Singh <[email protected]>
### What is this PR for?
Generic ConfInterpreter only support process level configuration, this PR is trying to introduce UserSessionConfInterpreter which can customize session level configuration, One example is for livy interpreter that can customize livy session.

### What type of PR is it?
[Feature ]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3355

### How should this be tested?
* Ci pass and manually test livy interpreter

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2883 from zjffdu/ZEPPELIN-3355 and squashes the following commits:

6708ef0 [Jeff Zhang] ZEPPELIN-3355. Support inline configuration for session

(cherry picked from commit a1e69ad)
Signed-off-by: Jeff Zhang <[email protected]>
The code in this PR enables embedding/interpolating Z variables into command strings passed to Spark's SQL and Shell interpreters. It implements the functionality described in issue [ZEPPELIN-1967](https://issues.apache.org/jira/browse/ZEPPELIN-1967)

    This PR resumes a fresh effort while taking into consideration all of the discussion in the context of the earlier [PR-2502](apache#2502). The earlier PR-2502 was closed due to a corruption in my repo that could not be corrected.

    The code in this PR resolves all of the discussion and suggestions in the body of the earlier [PR-2502](apache#2502). The following description is a summary of the current implementation:

    Patterns of the form `{var-name}` within commands will be interpolated only if a predefined object of the specified name exists in `z`. Additionally, all such patterns within the command line should also be translatable for any interpolation to occur. Partial translation of a command line (where only some of the patterns are translated, and others are not) is never performed.

    Patterns of the form `{{any-text}}` are translated into `{any-text}`. This feature is an escaping mechanism that allows `{` and `}` characters to be passed into a command without invoking the interpolation mechanism.

    The translations described above are performed only when all occurrences of `{`, `}`, `{{`, and `}}` in any command string conform to one of the two forms described above. A command that contains `{` and/or `}` characters used in any other way (than `{var-name}` and `{{any-text}}` as described above) will be used as-is without attempting any translations whatsoever -- even if the command also contains legal, translate-able and/or escape-able, constructs of the above two forms.

    No error is flagged in any case. This behavior is identical to the implementation of a similar feature in
    Jupyter's shell invocation using the ! magic command.

    At present only the SQL and Shell interpreters support object interpolation.

    [Improvement]

    * [ ] - Task

    https://issues.apache.org/jira/browse/ZEPPELIN-1967

    A new unit-test class ZeppCtxtVariableTest.java (see below) has been added. The attached screenshots below also show tests of the functionality.

    ![figure-1](https://user-images.githubusercontent.com/477015/36956999-5f8cca92-2057-11e8-8b76-f4ccd2a21d50.png)
    ![figure-2](https://user-images.githubusercontent.com/477015/36957001-650f271c-2057-11e8-8e94-4805fd24e796.png)
    ![figure-3](https://user-images.githubusercontent.com/477015/36957005-6a747dec-2057-11e8-9c72-4ebef17b52db.png)
    ![figure-4](https://user-images.githubusercontent.com/477015/36957006-6df95ad2-2057-11e8-8585-3eb679e3a146.png)
    ![figure-5](https://user-images.githubusercontent.com/477015/36957011-7284be02-2057-11e8-9204-3774121397e6.png)

    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? Yes, and detailed documentation has been added to the part describing ZeppelinContext variables (see file spark.md below).

Author: Sanjay Dasgupta <[email protected]>

Closes apache#2898 from zjffdu/ZEPPELIN-1967-0.8 and squashes the following commits:

800e3c5 [Sanjay Dasgupta] [ZEPPELIN-1967] Passing Z variables to Shell and SQL Interpreters
…table data

### What is this PR for?
z.getInterpreterContext().out().clear() fails to clear table data

### What type of PR is it?
Bug Fix

### Todos
* [ ] - Fix visualizations in a separate ticket

### What is the Jira issue?
[ZEPPELIN-3313](https://issues.apache.org/jira/browse/ZEPPELIN-3313)

### How should this be tested?
see jira description

### Screenshots (if appropriate)

#### before
![before](https://user-images.githubusercontent.com/2031306/37593485-2585fb74-2b98-11e8-9a21-8dd81a0c4c7d.gif)

#### after
![after](https://user-images.githubusercontent.com/2031306/37593495-2e106ba8-2b98-11e8-9dab-80985b82ccec.gif)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Renjith Kamath <[email protected]>

Closes apache#2881 from r-kamath/ZEPPELIN-3313 and squashes the following commits:

d67974e [Renjith Kamath] ZEPPELIN-3313 z.getInterpreterContext().out().clear() fails to clear table data

(cherry picked from commit fd27014)
Signed-off-by: Renjith Kamath <[email protected]>
### What is this PR for?
The original purpose of apache#2158 was correct processing of ';'. This was done via full removing comments from code.
Unfortunately Apache Phoenix uses hooks in comments https://forcedotcom.github.io/phoenix/#hintml.
Thus we should not delete comments in scripts.
There was discussion about comment rules for different databases (solr). The right way is keep style. Thus analysts can copy queries to another tool and can get same results and errors.

### What type of PR is it?
[Bug Fix]

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3344

### How should this be tested?
* Unit tests pass: testSplitSqlQueryWithComments and testSplitSqlQuery

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: mebelousov <[email protected]>

Closes apache#2876 from mebelousov/ZEPPELIN-3344 and squashes the following commits:

6980400 [mebelousov] ZEPPELIN-3344 Fix checkstyle, add tests for comments
83c8e8f [mebelousov] ZEPPELIN-3344 Rename test
eed54c8 [mebelousov] ZEPPELIN-3344 Revert comments in JDBC interpreter

(cherry picked from commit 8238b71)
Signed-off-by: Jeff Zhang <[email protected]>
…preter support completion

This is what I see in zeppelin and jupyter. The difference is due to zeppelin also add code completion of ace editor. I think code completion of ace editor should only be enabled when the backend interpreter doesn't support completion.

[Bug Fix]

* [ZEPPELIN-3001](https://issues.apache.org/jira/browse/ZEPPELIN-3001)

* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A

Author: Prabhjyot Singh <[email protected]>

Closes apache#2885 from prabhjyotsingh/ZEPPELIN-3001 and squashes the following commits:

fcd551f [Prabhjyot Singh] meta can be either empty string or undefined
e6ad8d1 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-3001
508cd23 [Prabhjyot Singh] fix websocketMsgSrv.completion paragraph id issue
9380c32 [Prabhjyot Singh] upgrade ace-builds to 1.3.2
701f93f [Prabhjyot Singh] make default suggestion if there is no suggestion from back-end.
97f594d [Prabhjyot Singh] completionSupport false for SparkRInterpreter
d2f70ae [Prabhjyot Singh] get completionSupport from interpreter (backend)
adaabcb [Prabhjyot Singh] ZEPPELIN-3001: Disable ace editor code completion when back-end interpreter support completion

Change-Id: I03e39422bafebf9c96b01b88f542a16dd6959f23
(cherry picked from commit fe07e5a)
Signed-off-by: Prabhjyot Singh <[email protected]>
### What is this PR for?
Trivial fix for javadoc of branch-0.8

### What type of PR is it?
[Hot Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* No jira created

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? NO

Author: Jeff Zhang <[email protected]>

Closes apache#2906 from zjffdu/JavaDoc_0.8 and squashes the following commits:

61ccd4d [Jeff Zhang] [HOTFIX] JavaDoc Fix for branch-0.8
…nEnabled is not set true

### What is this PR for?
Simply fixes NPE by checking for null

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3332

### Questions:
Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

Author: bag_of_tricks <[email protected]>

Closes apache#2911 from felixalbani/ZEPPELIN-3332 and squashes the following commits:

c6ccf41 [bag_of_tricks] ZEPPELIN-3332
622333e [bag_of_tricks] Revert "ZEPPELIN-3332"
dd7d37d [bag_of_tricks] ZEPPELIN-3332

(cherry picked from commit 685eb92)
Signed-off-by: Lee moon soo <[email protected]>
### What is this PR for?
This PR just does a quick fix this security issue.
1. Remove the setting cron user in frontend
2. Run the note via owner.

### What type of PR is it?
[Bug Fix ]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3350

### How should this be tested?
* Manually tested

### Screenshots (if appropriate)
![screen shot 2018-04-09 at 3 04 06 pm](https://user-images.githubusercontent.com/164491/38483974-707dca56-3c07-11e8-918a-cd47ed94ee99.png)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2914 from zjffdu/ZEPPELIN-3350 and squashes the following commits:

63325d0 [Jeff Zhang] [ZEPPELIN-3350] Don't allow set cronExecutionUser

(cherry picked from commit 08c9ad9)
Signed-off-by: Jeff Zhang <[email protected]>
### What is this PR for?
Zeppelin npm error: getaddrinfo ENOTFOUND
- upgrade frontend-maven-plugin to 1.4

### What type of PR is it?
Bug Fix

### Todos
* [ ] - Task

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3400

### Questions:
* Does the licenses files need update? n/a
* Is there breaking changes for older versions? n/a
* Does this needs documentation? n/a

Author: Renjith Kamath <[email protected]>

Closes apache#2921 from r-kamath/ZEPPELIN-3400 and squashes the following commits:

3326f12 [Renjith Kamath] Zeppelin npm error: getaddrinfo ENOTFOUND

(cherry picked from commit 2ba75c7)
Signed-off-by: Renjith Kamath <[email protected]>
…rFromKeytab` in JDBC interpreter

In JDBC interpreter every time `getConnection` has requested a call is made to `UserGroupInformation.loginUserFromKeytab` this PR is created to reduce the number of time it gets called.

[Improvement]

* [ZEPPELIN-3384](https://issues.apache.org/jira/browse/ZEPPELIN-3384)

* Does the licenses files need update? n/a
* Is there breaking changes for older versions? n/a
* Does this needs documentation? n/a

Author: Prabhjyot Singh <[email protected]>

Closes apache#2909 from prabhjyotsingh/ZEPPELIN-3384 and squashes the following commits:

72b16be [Prabhjyot Singh] add more log
b0be207 [Prabhjyot Singh] ZEPPELIN-3384: Reduce number of calls that are made to `UserGroupInformation.loginUserFromKeytab`

Change-Id: I5d6265bcfddf32d84b22a520475caf290efcba44
(cherry picked from commit 87e5be9)
Signed-off-by: Prabhjyot Singh <[email protected]>
Zeppelin documentation not pointing to correct URL.

Zeppelin documentation is redirecting to http://zeppelin.apache.org/docs//index.html and this URL doesn't exist.

The available doc for 0.8.0 is http://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/index.html

[Bug Fix]

* [ZEPPELIN-3403](https://issues.apache.org/jira/browse/ZEPPELIN-3403)

* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A

Author: Prabhjyot Singh <[email protected]>

Closes apache#2922 from prabhjyotsingh/ZEPPELIN-3403 and squashes the following commits:

beb2277 [Prabhjyot Singh] ZEPPELIN-3403: Zeppelin documentation not pointing to correct url

Change-Id: Ied8426d5291073bb787e1766a07ba87d592caca2
(cherry picked from commit ff4c0f3)
Signed-off-by: Prabhjyot Singh <[email protected]>
With apache#2909 one of the indentation rule was missed which is causing the build to break.

[Hot Fix]

* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A

Author: Prabhjyot Singh <[email protected]>

Closes apache#2927 from prabhjyotsingh/hotfix/ZEPPELIN-3384 and squashes the following commits:

f4a5141 [Prabhjyot Singh] ZEPPELIN-3384: Checkstyle violation.

Change-Id: I9bbb503307e9b04b8ed351a213bc41de097e2ece
(cherry picked from commit 440701d)
Signed-off-by: Prabhjyot Singh <[email protected]>

# Conflicts:
#	jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
### What is this PR for?
Travis build failure due to npm error
e.g: https://travis-ci.org/TinkoffCreditSystems/zeppelin/jobs/365002217

### What type of PR is it?
Bug Fix

### Todos
* [ ] - Task

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3406

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Renjith Kamath <[email protected]>

Closes apache#2928 from r-kamath/ZEPPELIN-3406 and squashes the following commits:

14ca6be [Renjith Kamath] ZEPPELIN-3406 Travis build failure due to npm error

(cherry picked from commit 7aa94ce)
Signed-off-by: Renjith Kamath <[email protected]>
We have a precode for interpreter (all interpreters) which executes once after start interpreter. With it we can create shared code that will be available in the paragraph, create tables in the database and etc.
This precode for jdbc only which executes before run paragraph in same session (sql). In this precode we can declare session variables that will be available when you run paragraph (any sql which will be executed before the paragraph). For example `set application_name='#{user};`
Previous discussion in apache#2442

Feature

https://issues.apache.org/jira/browse/ZEPPELIN-2679

(jdbc interpreter, postgres)
1. set property **default.statementPrecode** `set application_name='#{user}';`
2. run paragraph `select current_setting('application_name');`

* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: tinkoff-dwh <[email protected]>
Author: mebelousov <[email protected]>
Author: Tinkoff DWH <[email protected]>
Author: Renjith Kamath <[email protected]>
Author: isys.mreshetov <[email protected]>

Closes apache#2920 from mebelousov/ZEPPELIN-2679 and squashes the following commits:

d610805 [mebelousov] Revert "ZEPPELIN-3406 Travis build failure due to npm error"
11cbc44 [Renjith Kamath] ZEPPELIN-3406 Travis build failure due to npm error
11f89a5 [mebelousov] ZEPPELIN-2679 Small fix in docs
e9d36d2 [mebelousov] Merge remote-tracking branch 'apache/master' into ZEPPELIN-2679
00aeabf [tinkoff-dwh] ZEPPELIN-2679 fix checkstyle
f7529ca [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2679
07b41c5 [tinkoff-dwh] ZEPPELIN-2679 rename test methods
cd0d6ed [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2679
7bba0db [tinkoff-dwh] ZEPPELIN-2679 rename session to statement
719d0a2 [isys.mreshetov] ZEPPELIN-2679 docs
ae7a0d6 [Tinkoff DWH] [ZEPPELIN-2679] tests
bd3d809 [Tinkoff DWH] [ZEPPELIN-2679] jdbc. precode for session

(cherry picked from commit edf3c25)
Signed-off-by: Jeff Zhang <[email protected]>
…tion

### What is this PR for?
Fix impersonate spark interpreter.
Currently, impersonate spark interpreter always runs as zeppelin user, not impersonated user when disable proxy-user option.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3395

### How should this be tested?
1. ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=false in zeppelin-env.sh
2. set up user impersonation flag
3. run some job using that spark interpreter
4. spark interpreter process should be created with currently logged-in user

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: iijima_satoshi <[email protected]>

Closes apache#2916 from iijima-satoshi/fix-impersonate-spark and squashes the following commits:

a7465b6 [iijima_satoshi] Fix: Spark interpreter does not run as impersonated user when disable proxy-user option

(cherry picked from commit aa3ea68)
Signed-off-by: Jeff Zhang <[email protected]>
…onf' interpreters

### What is this PR for?
The root cause of this bug is that the final status of paragraph is not only determined by interpreter, but also by its scheduler. Here the bug is in FIFOScheduler. I just fix it as a quick solution in this PR. But for a long term solution, I think we may need to do code refactoring in the scheduler component.

### What type of PR is it?
[Bug Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3391

### How should this be tested?
* Manually tested

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2926 from zjffdu/ZEPPELIN-3391 and squashes the following commits:

09441ad [Jeff Zhang] ZEPPELIN-3391. Incorrect status shown for '%livy2.conf' and %spark2.conf' interpreters

(cherry picked from commit 0e82a57)
Signed-off-by: Jeff Zhang <[email protected]>
### What is this PR for?
Use system PYTHONPATH in spark.pyspark interpreter, if PYTHONPATH already exists in environment variables.

### What type of PR is it?
[Bug Fix]

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3347

### How should this be tested?
* Add some directories to PYTHONPATH
`export PYTHONPATH="/opt/python/dir1:/opt/python/dir2:$PYTHONPATH"`
* Build branch "ZEPPELIN-3347_0.8"
* Run in %spark.pyspark
```
import sys
print('\n'.join(sys.path))
```
* Check that the directories are in the output list.
Compare with the result in %python.

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>
Author: mebelousov <[email protected]>

Closes apache#2907 from mebelousov/ZEPPELIN-3347_0.8 and squashes the following commits:

7200746 [Jeff Zhang] [HOTFIX] JavaDoc Fix for branch-0.8
8ada1e3 [mebelousov] ZEPPELIN-3347 Fix "PYTHONPATH" for spark.pyspark
### What is this PR for?

The root cause is that we may do UserGroupInformation.loginUserFromKeytab multiple times if we use hdfs for several places, e.g. for notebook repo and zeppelin config. This PR fix the bug by only doing UserGroupInformation.loginUserFromKeytab one time.

### What type of PR is it?
[Bug Fix ]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3356

### How should this be tested?
Manually tested
* Add the following properties to /etc/krb5.conf
   renew_lifetime = 10m
  ticket_lifetime = 5m
* Add the following properties to /var/kerberos/krb5kdc/kdc.conf
  max_renewable_life = 10m
  max_life = 5m

Without this PR, will hit the ticket expire after 10 minutes. With this PR, the ticket is still valid after 10 minutes

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2924 from zjffdu/ZEPPELIN-3356 and squashes the following commits:

00eda00 [Jeff Zhang] [ZEPPELIN-3356] Kerberos ticket still expire after 7 days

(cherry picked from commit 6cb39a0)
Signed-off-by: Jeff Zhang <[email protected]>
…longs to roles with space in its name

- escape roles value in login and ticket json response
- fix method name typo

Fix string escape issue in roles. More details in JIRA description.

Bug Fix

* [ ] - Task

https://issues.apache.org/jira/browse/ZEPPELIN-3405

see JIRA description

* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Renjith Kamath <[email protected]>

Closes apache#2931 from r-kamath/ZEPPELIN-3405 and squashes the following commits:

87f8243 [Renjith Kamath] ZEPPELIN-3405 Zeppelin fails to display the User home page if user belongs to roles with space in its name.

Change-Id: I58be55c2a663f5dfddeab990654d11e5088de1ec
(cherry picked from commit 4013259)
Signed-off-by: Renjith Kamath <[email protected]>
Export functionality for both notebook and download data as CSV stopped working for Chrome.

[Bug Fix]

* [ZEPPELIN-3415](https://issues.apache.org/jira/browse/ZEPPELIN-3415)

* Try exporting notebook and/or download data as CSV for any table.

Before:
![download button before](https://user-images.githubusercontent.com/674497/38939688-1d397dd6-4346-11e8-8738-d3d0a00dc333.gif)

After:
![download button after](https://user-images.githubusercontent.com/674497/38939687-1d04681c-4346-11e8-8183-c46042ede815.gif)

* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A

Author: Prabhjyot Singh <[email protected]>

Closes apache#2932 from prabhjyotsingh/ZEPPELIN-3415 and squashes the following commits:

fae64e0 [Prabhjyot Singh] ZEPPELIN-3415: Fix export notebook functionality

Change-Id: Iad481bc304c6dd2d8c6fcb22cd4f4287c89b8d39
(cherry picked from commit 2472e2d)
Signed-off-by: Prabhjyot Singh <[email protected]>
This PR introduces a new configuration property to convert username to lower case. This is useful when the users (from external sources like AD/LDAP) are coming in with mixed-case names and Hadoop services (like Hive) can't authorize them correctly because Hadoop services recognize users only in lower case (like Linux).

Adding a new config option "zeppelin.username.force.lowercase" to handle such scenarios.

Behavior without this PR:
Access is denied to CaMel case user while running a Hive paragraph

Behavior with this PR:
User is allowed to run query when proposed configuration set to true.
By default, keeping zeppelin.username.force.lowercase=false to retain the current behavior.

[Bug Fix]

* [ ] - Unit test

* https://issues.apache.org/jira/browse/ZEPPELIN-3312

* Travis CI should pass
* Manual steps to test:
1. Configure Zeppelin with Active Directory authentication
2. Login to Zeppelin as a CaMel case user
3. Try to run a simple JDBC note with a Hive query (like a select * query). This would fail with "user [CaMel] does not have proper privileges to [USE] operation" error message.
4. Now set zeppelin.username.force.lowercase=true in custom zeppelin-site.xml configuration.
5. Once again, login as CaMel case user. This time the same Hive query would run as expected. Because the username is now passed in lower case.
6. Also notice that after successful login, the login username (in the top-right corner) will be in lower case too.

* Login as CaMel case user:
<img width="596" alt="screen shot 2018-04-12 at 3 33 01 am" src="https://user-images.githubusercontent.com/15668387/38672744-faf00f8e-3e03-11e8-86b2-cc5981d380d2.png">
* Notice the converted username post login:
<img width="806" alt="screen shot 2018-04-12 at 3 33 43 am" src="https://user-images.githubusercontent.com/15668387/38672777-108c7b66-3e04-11e8-8c97-467b4b73fe3d.png">

* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Vipin Rathor <[email protected]>

Closes apache#2923 from VipinRathor/ZEPPELIN-3312 and squashes the following commits:

886acb9 [Vipin Rathor] Add lowercase username support for interpreter permission
c112098 [Vipin Rathor] Remove maven cyclic refernce and use conf object directly
549f84d [Vipin Rathor] Convert username for Notebook Authorization as well
f83ce9f [Vipin Rathor] Adding new test testUsernameForceLowerCase and fixing canGetPrincipalName
b272299 [Vipin Rathor] Fixing Travis CI build failure due to indentation
83fb686 [Vipin Rathor] Incorporating PR review suggestion to zeppelin-site.xml-template
7f9b4df [Vipin Rathor] Add support to force username case conversion

Change-Id: Id13a5eca9718063cb629454a9d3d2fc6f3cfb663
(cherry picked from commit b89c9ad)
Signed-off-by: Prabhjyot Singh <[email protected]>

# Conflicts:
#	zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
#	zeppelin-server/src/test/java/org/apache/zeppelin/security/SecurityUtilsTest.java
…eterSettingManager

### What is this PR for?
Removing redundant synchronized code to avoid blocking other logics.

### What type of PR is it?
[Bug Fix]

### Todos
* [x] - Change synchronized block to read/write lock

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3411

### How should this be tested?
* Current tests should be passed

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jongyoul Lee <[email protected]>

Closes apache#2935 from jongyoul/ZEPPELIN-3411 and squashes the following commits:

3b90155 [Jongyoul Lee] Removed unused `import` statements Simplified `for` loop
4691301 [Jongyoul Lee] Removed lock/synchronized codes because interpreterSettings already is implemented by currentHashMap
24be692 [Jongyoul Lee] Removed all synchronized blocks and replace them to read/write lock
Update xercesImpl to 2.11.0.SP5 to get some fixes. This is a required dependency for jdbc-phoenix.

[ Improvement ]

* [ZEPPELIN-3416](https://issues.apache.org/jira/browse/ZEPPELIN-3416)

* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A

Author: Prabhjyot Singh <[email protected]>

Closes apache#2933 from prabhjyotsingh/ZEPPELIN-3416 and squashes the following commits:

54677f1 [Prabhjyot Singh] bump version of beanutils to 1.9.2
8c891b6 [Prabhjyot Singh] ZEPPELIN-3416: Update xercesImpl to 2.11.0.SP5

Change-Id: I592b7e8dbf56144d65d5ba726d92f230d51b49ac
(cherry picked from commit 322c868)
Signed-off-by: Prabhjyot Singh <[email protected]>
hotfix for apache#2931
apache#2931 (comment)

[Hot Fix]

* [ ] - Task

https://issues.apache.org/jira/browse/ZEPPELIN-3405

see PR description

* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Renjith Kamath <[email protected]>

Closes apache#2942 from r-kamath/hotfix_ZEPPELIN-3405 and squashes the following commits:

b93c2de [Renjith Kamath] ZEPPELIN-3405 fix permission checks with roles

Change-Id: I1ce0d0306dc665b0267042a5e9181a13f136fac1
(cherry picked from commit ad77265)
Signed-off-by: Renjith Kamath <[email protected]>
Just fix a typo

Hot Fix

* Not sure if a jira issue is required?

* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Thomas Decaux <[email protected]>

Closes apache#2940 from ebuildy/patch-1 and squashes the following commits:

7a4ebab [Thomas Decaux] Fix type note => user

Change-Id: If7c5afd42fa162d3337386408d4722062d5eb59b
(cherry picked from commit c4fdf58)
Signed-off-by: Renjith Kamath <[email protected]>
… expires (KnoxJwtRealm)

Zeppelin should ask user to re-login after JWT cookie expires.

[Bug Fix]

* [ZEPPELIN-3311](https://issues.apache.org/jira/browse/ZEPPELIN-3311)

Steps to reproduce :
1) Enable SSO for zeppelin
2) Login to zeppelin using SSO.
3) Now wait till JWT expires(knoxsso.token.ttl under Advanced knoxsso-topology could be set to lower values to get a short lived cookie)
4) Refresh zeppelin. Zeppelin doesn't ask for re-login, though JWT is already expired and not valid anymore.

* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A

Author: Prabhjyot Singh <[email protected]>

Closes apache#2941 from prabhjyotsingh/ZEPPELIN-3311 and squashes the following commits:

0014d5a [Prabhjyot Singh] ZEPPELIN-3311: Zeppelin should ask user to re-login after JWT cookie expires (KnoxJwtRealm)

Change-Id: Ie113f6078974cf914d259fae3d2987cf8364d71c
(cherry picked from commit 5632d2c)
Signed-off-by: Renjith Kamath <[email protected]>
### What is this PR for?

SparkInterpreterLauncherTest will fail when you already have enviroment variable setup in the machine where you run the unit test.

### What type of PR is it?
[Bug Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3428

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <[email protected]>

Closes apache#2943 from zjffdu/ZEPPELIN-3428 and squashes the following commits:

6b89b22 [Jeff Zhang] ZEPPELIN-3428. Flaky test: SparkInterpreterLauncherTest

(cherry picked from commit 8068314)
Signed-off-by: Jeff Zhang <[email protected]>
idzikovskyi and others added 29 commits October 30, 2019 17:08
(cherry picked from commit 54af51e)
…ge is enabled

(cherry picked from commit ea656bc)

  #Squashed
    MZEP-167 Support only verified and working versions of Helium plugins
(cherry picked from commit 4528c7b)
(cherry picked from commit b5f7563)
(cherry picked from commit ce22d43)

  #Squashed
    DSR-88 Refactor Spark MapR-DB Binary tutorials notebook according to ZEPPELIN-3587 (apache#132)
DSR-94 Update list of Helium plugins to recent version
MZEP-182 Install git package in DSR images
mapr-devops pushed a commit that referenced this pull request May 8, 2025
…Spark Interpreter Scala REPL and fix the CI failure due to low Scala version

### What is this PR for?
- fix the [CI failure](https://travis-ci.org/github/apache/zeppelin/builds/709913046) due to [PR-3852](apache#3852)

### What type of PR is it?
[Bug Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* [ZEPPELIN-4962](https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-4962)

### How should this be tested?
* CI test

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? Yes

Author: xiejiajun <[email protected]>
Author: xie-jia-jun <[email protected]>
Author: JakeXie <[email protected]>

Closes apache#3860 from xiejiajun/ZEPPELIN-4962 and squashes the following commits:

9128c9b [JakeXie] spark.repl.target docs update
ad4c0e3 [xiejiajun] Clear irrelevant code
a12d3a9 [xiejiajun] Support for manually specifying the Java version of Spark Interpreter Scala REPL and fix the CI failure due to low Scala version
ab2b191 [xiejiajun] Merge branch 'master' of https://github.com/apache/zeppelin into apache-master
5569788 [xiejiajun] Merge branch 'master' of https://github.com/apache/zeppelin into apache-master
0a9af6c [xiejiajun] Merge branch 'master' of https://github.com/apache/zeppelin into apache-master
be36b37 [xiejiajun] 合并Apache Master分支冲突解决
1335d55 [xiejiajun] Merge remote-tracking branch 'origin/master'
fc59f57 [JakeXie] Merge pull request apache#4 from apache/master
9cc70fe [xiejiajun] Merge remote-tracking branch 'origin/master'
6ef9b23 [xie-jia-jun] Merge pull request apache#3 from apache/master
45af87a [xiejiajun] added timeout for getting Thrift client to avoid situations where the interpreter may not be restarted when the interpreter process exits unexpectedly
f149c3b [xie-jia-jun] Merge pull request #1 from apache/master
5d4b645 [xie-jia-jun] Support OSSConfigStorage of Aliyun
dbb6639 [xie-jia-jun] Add Aliyun OSS SDK
bb47849 [xie-jia-jun] Support S3ConfigStorage of AWS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.