Rqueue job checkin (#54)

Job Checkin
Long running jobs can do checkin periodically using job.checkin("I'm doing something");
Display past job history for a message in dashboard.
Delete message metadata along with messages using background job
Allow deleting messages from normal and scheduled queues instead of only dead letter queue.
Fixes Redis string deserialisation issue, string were inserted without quote''
Fixes CSP rule error for inline javascript
Scan only required beans for RqueueListener annotated methods
Fixed double minus sign (--) in dashboard.
Potential error for periodic message, if period was longer than 24 hours
Dead letter queue message at front

Author: sonus21

.circleci/config.yml

@@ -18,11 +18,10 @@ executors:
       ORG_GRADLE_PROJECT_sonatypePassword: xxx
       USER_NAME: rqueue
       REDIS_RUNNING: "true"
-    run:
 
 
 jobs:
-  boot_2_1:
+  build:
     executor: rqueue-executor
     environment:
       SPRING_BOOT_VERSION: 2.1.0.RELEASE
@@ -32,27 +31,102 @@ jobs:
 
     steps:
       - checkout
-
-      # Download and cache dependencies
       - restore_cache:
           keys:
             - v1-dependencies-{{ checksum "build.gradle" }}
             # fallback to using the latest cache if no exact match is found
             - v1-dependencies-
 
       - run: gradle dependencies
-
       - save_cache:
           paths:
             - ~/.gradle
           key: v1-dependencies-{{ checksum "build.gradle" }}
+      - run: ./gradlew compileJava
+      - run: ./gradlew compileTestJava
+      - persist_to_workspace:
+          root: .
+          paths:
+            - .
+  unit_test:
+    executor: rqueue-executor
+    environment:
+      SPRING_BOOT_VERSION: 2.1.0.RELEASE
+      SPRING_VERSION: 5.1.2.RELEASE
+      SPRING_DATA_VERSION: 2.1.2.RELEASE
+      MICROMETER_VERSION: 1.1.0
+    steps:
+      - attach_workspace:
+          at: .
+      - run: ./gradlew codeCoverageReport -DincludeTags=unit
+      - persist_to_workspace:
+          root: .
+          paths:
+            - .
+  producer_only_test:
+    executor: rqueue-executor
+    environment:
+      SPRING_BOOT_VERSION: 2.1.0.RELEASE
+      SPRING_VERSION: 5.1.2.RELEASE
+      SPRING_DATA_VERSION: 2.1.2.RELEASE
+      MICROMETER_VERSION: 1.1.0
+    steps:
+      - attach_workspace:
+          at: .
       - run:
-         name: Install Dependencies
-         command: |
-          sudo add-apt-repository ppa:chris-lea/redis-server
-          sudo apt-get update
-          sudo apt-get -y install redis-server
-          redis-cli --version
+          name: Install Dependencies
+          command: |
+            sudo add-apt-repository ppa:chris-lea/redis-server
+            sudo apt-get update
+            sudo apt-get -y install redis-server
+            redis-cli --version
+      - run: ./gradlew codeCoverageReport -DincludeTags=producerOnly
+      - persist_to_workspace:
+          root: .
+          paths:
+            - .
+
+  integration_test:
+    executor: rqueue-executor
+    environment:
+      SPRING_BOOT_VERSION: 2.1.0.RELEASE
+      SPRING_VERSION: 5.1.2.RELEASE
+      SPRING_DATA_VERSION: 2.1.2.RELEASE
+      MICROMETER_VERSION: 1.1.0
+    steps:
+      - attach_workspace:
+          at: .
+      - run:
+          name: Install Dependencies
+          command: |
+            sudo add-apt-repository ppa:chris-lea/redis-server
+            sudo apt-get update
+            sudo apt-get -y install redis-server
+            redis-cli --version
+      - run: ./gradlew codeCoverageReport -DincludeTags=integration -DexcludeTags=redisCluster,producerOnly
+      - persist_to_workspace:
+          root: .
+          paths:
+            - .
+
+  redis_custer_test:
+    executor: rqueue-executor
+    environment:
+      SPRING_BOOT_VERSION: 2.1.0.RELEASE
+      SPRING_VERSION: 5.1.2.RELEASE
+      SPRING_DATA_VERSION: 2.1.2.RELEASE
+      MICROMETER_VERSION: 1.1.0
+    steps:
+      # Reuse the workspace from the build job
+      - attach_workspace:
+          at: .
+      - run:
+          name: Install Dependencies
+          command: |
+            sudo add-apt-repository ppa:chris-lea/redis-server
+            sudo apt-get update
+            sudo apt-get -y install redis-server
+            redis-cli --version
       - run:
           name: Setup Redis Cluster
           background: true
@@ -72,47 +146,41 @@ jobs:
             cd 9005 && redis-server ./redis.conf &
             sleep 30
             redis-cli --cluster create 127.0.0.1:9000 127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003 127.0.0.1:9004 127.0.0.1:9005 --cluster-replicas 1 --cluster-yes
-
-      - run: ./gradlew codeCoverageReport
-      - run:
-          name: Build Success
-          when: on_success
-          command: ./gradlew coveralls
-
-  boot_2_2:
+      - run: ./gradlew codeCoverageReport -DincludeTags=redisCluster
+      - persist_to_workspace:
+          root: .
+          paths:
+            - .
+  report_code_coverage:
     executor: rqueue-executor
     environment:
-      SPRING_BOOT_VERSION: 2.2.0.RELEASE
-      SPRING_VERSION: 5.2.0.RELEASE
-      SPRING_DATA_VERSION: 2.2.0.RELEASE
-      MICROMETER_VERSION: 1.3.2
+      SPRING_BOOT_VERSION: 2.1.0.RELEASE
+      SPRING_VERSION: 5.1.2.RELEASE
+      SPRING_DATA_VERSION: 2.1.2.RELEASE
+      MICROMETER_VERSION: 1.1.0
     steps:
-      - checkout
-
-      # Download and cache dependencies
-      - restore_cache:
-          keys:
-            - v2-dependencies-{{ checksum "build.gradle" }}
-            # fallback to using the latest cache if no exact match is found
-            - v2-dependencies-
-
-      - run: gradle dependencies
-
-      - save_cache:
-          paths:
-            - ~/.gradle
-          key: v2-dependencies-{{ checksum "build.gradle" }}
-
-      # run tests!
-      - run: ./gradlew codeCoverageReport
-      - run:
-          name: Upload Code Coverage Data
-          when: always #on_success
-          command: ./gradlew coveralls
-
+      - attach_workspace:
+          at: .
+      - run: ls -r build/reports/jacoco/test
+      - run: pwd
+      - run: ./gradlew coveralls
 
 workflows:
   main:
     jobs:
-      - boot_2_1
-      #- boot_2_2
+      - build
+      - unit_test:
+          requires:
+            - build
+      - producer_only_test:
+          requires:
+            - unit_test
+      - integration_test:
+          requires:
+            - producer_only_test
+      - redis_custer_test:
+          requires:
+            - integration_test
+      - report_code_coverage:
+          requires:
+            - redis_custer_test

CHANGELOG.md

@@ -1,17 +1,43 @@
 # [Rqueue] New and Notable Changes
 
-**NOTE**: The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [2.3.0] - 2-Jan-2021
+
+### Added
+
+* Job checkin for long-running tasks
+* Display job and failure details in UI for each message
+* Allow deleting messages from normal and scheduled queues instead of only dead letter queue.
+* Scan only required beans for RqueueListener annotated methods
+
+### Fixes
+
+* Redis string deserialization issue, string were inserted without quote''
+* Dashboard CSP rule error for inline javascript
+* Double minus sign (--) in UI
+
+### Miscellaneous
+
+* Delete message metadata along with messages using background job
+* Potential error for a periodic message, if period was longer than 24 hours
+* Add retry limit exceeded messages at the front of dead letter queue instead at the back.
 
 ## [2.2.0] - 6-Dec-2020
+
 ### Added
-* New API to enqueue periodic message. Periodic jobs are like cron jobs that would run at the certain interval.
+
+* New API to enqueue periodic message. Periodic jobs are like cron jobs that would run at the
+  certain interval.
 
 ## [2.1.1] - 24-Sep-2020
+
 ### Added
+
 * More apis to enqueue unique message
 
 ## [2.1.0] - 16-Sep-2020
+
 ### Added
+
 * Allow application to provide message id while enqueuing messages
 * Unique message enqueue
 * Api to check if message was enqueued or not
@@ -20,37 +46,43 @@
 * Enqueue list of objects and process them, like batch-processing
 
 Fixes:
+
 * Registered queues should not be deleted when used in producer mode
 
 ## [2.0.4] - 2-Aug-2020
 
 ### Added
-- Allow a listener to be added on dead letter queue 
+
+- Allow a listener to be added on dead letter queue
 
 ### Fixes:
-- Rqueue views/apis not accessible via api gateway
 
+- Rqueue views/apis not accessible via api gateway
 
 ## [2.0.2] - 13-July-2020
 
 ### Fixes
+
 - JDK dynamic proxy
 - AoP profiler
 
 ## [2.0.1] - 17-May-2020
 
 ### Added
+
 - Allow registering a queue, that can be in push only mode
-- Apis to schedule task at the given time 
+- Apis to schedule task at the given time
 - Refine enqueueIn apis to support Duration and TimeUnit
 
 ### Fixes
-- Arguments mismatch due to multiple class loaders.
-- Dead letter queue clear lead to clearing all the messages related to that queue. 
 
+- Arguments mismatch due to multiple class loaders.
+- Dead letter queue clear lead to clearing all the messages related to that queue.
 
 ## [2.0.0] - 10-May-2020
+
 ### Added
+
 - Web interface to visualize queue
 - Move message from one queue to another
 - Latency visualizer
@@ -64,53 +96,82 @@ Fixes:
 - Strict or weighted algorithm for message execution
 
 ### Breaking Changes
-- Queue names are prefixed, that can lead to error.  1.x users set REDIS key `__rq::version` to `1`. It does try to find the version using key prefix, but if all queues are empty or no key exist in REDIS with prefix `rqueue-` then it will consider version 2.
+
+- Queue names are prefixed, that can lead to error. 1.x users set REDIS key `__rq::version` to `1`.
+  It does try to find the version using key prefix, but if all queues are empty or no key exist in
+  REDIS with prefix `rqueue-` then it will consider version 2.
 - Renamed annotation field `maxJobExecutionTime` to `visibilityTimeout`
 
 ### Fixes
-- **Spring** Optional Micrometer, in older version config class was importing micrometer related classes, that could lead to error if classes are not found. In this version now code depends on bean name using DependsOn annotation.
+
+- **Spring** Optional Micrometer, in older version config class was importing micrometer related
+  classes, that could lead to error if classes are not found. In this version now code depends on
+  bean name using DependsOn annotation.
 
 ## [1.4.0] - 08-Apr-2020
+
 * Allow queue level configuration of job execution time.
 * Support to add Message processor for discard and dead letter queue
 
 ## [1.3.2] - 01-Apr-2020
+
 * Support lower version of spring 2.1.x
 
 ## [1.3.1] - 27-Feb-2020
+
 * **Fixed** Bootstrap issue due to optional dependencies of micrometer
 
 ## [1.3] - 11-Dec-2019
-* Expose multiple queue metrics using micrometer. (queue-size, delay queue size, processing queue size, dead letter queue size, execution counter, failure counter)
+
+* Expose multiple queue metrics using micrometer. (queue-size, delay queue size, processing queue
+  size, dead letter queue size, execution counter, failure counter)
 * An api to move messages from dead letter queue to other queue. (Any source queue to target queue).
 
 ### Fixed
-* An issue in the scheduler that's always scheduling job at the delay of 5 seconds. (this leads to messages are not copied from delayed queue to main queue on high load)
 
+* An issue in the scheduler that's always scheduling job at the delay of 5 seconds. (this leads to
+  messages are not copied from delayed queue to main queue on high load)
 
 ## [1.2] - 03-Nov-2019
-* Typo of *Later* to *Letter*
 
+* Typo of *Later* to *Letter*
 
 ## [1.1] - 02-Nov-2019
+
 * At least once message guarantee
 * Reduced ZSET calls
 * Lua script to make atomic operation
 
 ## [1.0] - 23-Oct-2019
+
 * The basic version of Asynchronous task execution using Redis for Spring and Spring Boot
 
 [1.0]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.0-RELEASE
+
 [1.1]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.1-RELEASE
+
 [1.2]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.2-RELEASE
+
 [1.3]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.3-RELEASE
+
 [1.3.1]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.3.1-RELEASE
+
 [1.3.2]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.3.2-RELEASE
+
 [1.4.0]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/1.4.0-RELEASE
+
 [2.0.0]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/2.0.0-RELEASE
+
 [2.0.1]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/2.0.1-RELEASE
+
 [2.0.2]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue/2.0.2-RELEASE
+
 [2.0.4]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue-core/2.0.4-RELEASE
+
 [2.1.0]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue-core/2.1.0-RELEASE
+
 [2.1.1]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue-core/2.1.1-RELEASE
+
 [2.2.0]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue-core/2.2.0-RELEASE
+
+[2.3.0]: https://repo1.maven.org/maven2/com/github/sonus21/rqueue-core/2.3.0-RELEASE