Is there a way to output a pretty report when running via command line?

user_9847386 0
Viewed 875

Is there a way to output pretty report when running Karate via the command line?

Running this:

java -jar karate-0.9.1.jar tests/*.feature

Produces this:

15:32:56.363 [main] INFO  com.intuit.karate.netty.Main - Karate 
version: 0.9.1
15:32:56.466 [main] INFO  com.intuit.karate.Runner - Karate version: 
0.9.1
15:32:56.868 [ForkJoinPool-1-worker-1] WARN  com.intuit.karate - 
skipping bootstrap configuration: could not find or read file: 
classpath:karate-config.js
15:32:57.065 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - 
request:
1 > GET http://localhost:8080/ping
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Host: localhost:8080
1 > User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_191)

15:32:57.114 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - 
response time in milliseconds: 47.35
1 < 200
1 < Content-Length: 18
1 < Content-Type: application/json; charset=utf-8
1 < Date: Wed, 16 Jan 2019 15:32:57 GMT
1 < X-Request-Id: ffa3630d-19a3-11e9-bbab-0242ac180007
{"message":"pong"}

15:32:57.138 [ForkJoinPool-1-worker-1] ERROR com.intuit.karate - 
assertion failed: path: $.message, actual: 'pong', expected: 
'pong2', reason: not equal
15:32:57.163 [ForkJoinPool-1-worker-1] INFO  
com.intuit.karate.Runner - <<fail>> feature 1 of 1: 
tests/ping.feature
---------------------------------------------------------
feature: tests/ping.feature
report: target/tests.ping.json
scenarios:  1 | passed:  0 | failed:  1 | time: 0.2609
---------------------------------------------------------
Karate version: 0.9.1
======================================================
elapsed:   0.70 | threads:    1 | thread time: 0.26
features:     1 | ignored:    0 | efficiency: 0.37
scenarios:    1 | passed:     0 | failed: 1
======================================================
failed features:
tests.ping: ping.feature:8 - path: $.message, actual: 'pong', 
expected: 'pong2', reason: not equal

Exception in thread "main" picocli.CommandLine$ExecutionException: 
there are test failures
    at 
com.intuit.karate.netty.Main$1.handleExecutionException 
(Main.java:118)
    at picocli.CommandLine.parseWithHandlers(CommandLine.java:1157)
    at com.intuit.karate.netty.Main.main(Main.java:124)

Which is more of a log than a report.

I know Karate generates a pretty HTML report, but this isn't really useful in a CI pipeline when you want to quickly identify what tests have failed.

Something simialar to this would be great:

Feature: 

  Scenario: 
      Before hook
    Given this step passes
      AfterStep hook
      After hook
      error (RuntimeError)
      ./features/step_definitions/output_steps.rb:11:in `After'

Failing Scenarios:
cucumber features/test.feature:2

1 scenario (1 failed)
1 step (1 passed)
karate
2 Answers

Here's what I suggest, if you really feel you need a cleaner report - please write a small utility that will process all the *.json files in the target/cucumber-html-reports folder. All the information you asked for - even line-numbers is contained in those JSON files.

You can use as a reference this feature request - where we added a JSON file to output the stats in an "automation friendly" way: https://github.com/intuit/karate/issues/561

Since you are the only one who has asked for this so far, it is unlikely that we take this up - unless we get some kind of community contribution. Hope that makes sense.

user_1434750 
     Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 6.165 sec <<< FAILURE!
[1:6] get all characters([getAllPeople])  Time elapsed: 0.007 sec  <<< ERROR!
com.intuit.karate.exception.KarateException: getAllPeople.feature:12 - path: $[0].birth_year, actual: '19BBY', expected: '#number', reason: not a number
    at ✽.And match each $.results[*] == char (getAllPeople.feature:12)

[1:6] get the first character([getFirstCharacter])  Time elapsed: 0.001 sec  <<< ERROR!
com.intuit.karate.exception.KarateException: getFirstCharacter.feature:12 - path: $.birth_year, actual: '19BBY', expected: '19BB', reason: not equal
    at ✽.And match $ == firstChar (getFirstCharacter.feature:12)


Results :

Tests in error: 
  [1:6] get all characters([getAllPeople]): getAllPeople.feature:12 - path: $[0].birth_year, actual: '19BBY', expected: '#number', reason: not a number
  [1:6] get the first character([getFirstCharacter]): getFirstCharacter.feature:12 - path: $.birth_year, actual: '19BBY', expected: '19BB', reason: not equal

Tests run: 2, Failures: 0, Errors: 2, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.588 s
[INFO] Finished at: 2019-01-16T17:05:13+01:00
[INFO] Final Memory: 19M/237M

I believe that with this display, it is easy to quickly identify which tests have failed, and why. It is harder in your example because of the Java exception, but the Karate results are actually easier to read than a lot of CLI reports.

user_107916390
Related