Serialized the analyzing tiers to persist state #32
Conversation
|
Also solved issue #18 with regex |
|
|
Passed the local test! |
|
Looks great will take a detailed look later! |
|
Sorry for the delay, will try to look at this today |
|
May need to configure the pod Manager to use a volume for storage |
|
|
| spec: | ||
| containers: | ||
| - image: benblamey/hom-impl-2.manager:latest | ||
| - image: haoyuan9654/hom-impl-2.manager:latest |
There was a problem hiding this comment.
it is maybe not necessary to commit changes like this.
There was a problem hiding this comment.
i understand its a bit of a muddle. we don't exactly have a great CI process here...
There was a problem hiding this comment.
- Docker build failed with the old Dockerfile, fixed with the latest release.
- Ubuntu hirsute is no longer supported since January 20, 2022, replaced with jammy. Solved the error: repository does not have a Release file (non-zero code:100). See the official post: https://fridge.ubuntu.com/2022/01/21/ubuntu-21-04-hirsute-hippo-end-of-life-reached-on-january-20-2022/ and a question post: https://askubuntu.com/questions/1420130/cannot-do-apt-update-ip-address-not-found-no-release-file
- It seems openjdk-16-jre is not supported by jammy, install openjdk-16-jre instead.
Understood. I make these changes to fix the docker image build problems
There was a problem hiding this comment.
Fair enough should upgrade them. I didn't realise it had got so old.... SNIC is still using Ubuntu 20.
| # install curl | ||
| RUN apt install -y curl | ||
| RUN apt install -y openjdk-16-jre | ||
| RUN apt install -y openjdk-18-jre |
| @@ -1,4 +1,6 @@ | |||
| FROM ubuntu:hirsute | |||
| FROM ubuntu:jammy | |||
There was a problem hiding this comment.
Why did this change?
I'm hesitant to update everything inside pods unless it is to fix an issue, it potentially creates more complexity.
| RUN kubectl version --client | ||
|
|
||
| COPY build/libs/hom-impl-2.manager-1.0-SNAPSHOT.jar output.jar | ||
| COPY hom-impl-2.manager-1.0-SNAPSHOT.jar output.jar |
There was a problem hiding this comment.
not sure. isn't the path here set by gradle?
|
|
||
| // Placeholder for existin Kafka stream repr. the input source for the system. | ||
| public class InputTier extends Tier { | ||
| public class InputTier extends Tier { |
There was a problem hiding this comment.
ideally want to avoid making changes like this
| "grep --extended-regexp --only-matching \"^\\s*\\\"def ([^(])+\\(\" *.ipynb | sed -E \"s/(.+):\\s+\\\"def (.+)\\(/\\1::\\2/\"" | ||
| }, new File(directory), null, true).stdOut; | ||
| // grep --extended-regexp --only-matching "^\s*\"\s*def\s*[_a-zA-Z]+\w*\([_a-zA-Z]+\w*\)" *.ipynb | sed -E "s/(.+):\s+\"def (.+)\(/\1,\2/" | ||
| "grep --extended-regexp --only-matching \"^\\s*\\\"def\\s*[_a-zA-Z]+\\w*\\([_a-zA-Z]+\\w*\\)\" *.ipynb | sed -E \"s/(.+):\\s+\\\"def (.+)\\(/\\1::\\2/\"" |
There was a problem hiding this comment.
Lol!
Yes this felt like a bit of a hack at the time....
|
@haoyuan-l |
Great, thanks! |
|
Sorry @haoyuan-l that was not all the changes |
|
I think it is simpler to have one file. Lists should support serialization (this may need to setup somehow I can't remember exactly). Then we just serialize the list of tiers. We do this each time we change anything. We deserialize when starting the pod. |
2 participants
Exist problems with gradle-tasks execution. The docker image and JAR file need to be updated.