Declarative deployment for AEM applicationThanhBlockedUnblockFollowFollowingMar 12The objective of this article is to describe how to create an AEM-CLI (AEM Command Line Interface) and use it for Declarative Deployment to an AEM application with a lot of practical snippets of codes.
Article Version: 1.
01) IntroductionWhat and Why Declarative is better than procedural deployment?Procedural deployment describes a bundle of steps requires to transform the application from a particular version to another.
The main point of this practice is that it tightly depends on the context of each time deployment and would be executed differently based on the release requirements of a specific version of the application as well as the targeting version that application would need to transform to.
For example, let’s assume our AEM environment has been installed initially with some packages like: The Service Packs, Hotfixes, ACS-Common, Grabbit, Access Control Tool, and some custom packages.
In a particular window, our team would have released a new version for one of the custom packages or need to increase the version for one of the above third-party packages and would need to deploy those new changes to the server.
By using procedural deployment, we need create a document to describe clearly steps of how to deploy just those new packages in desired order to AEM and send it to the operation team to execute it.
Or we could properly go further to create a bash script to automate the process by using some CURL commands with pointing to our desired packages.
However, in next release, we would have to modify either this deployment process document or script for a different scenario deployment with different released packages.
Then one of the biggest disadvantages of procedural approach is that sometime in far future it is almost not possible to know which exactly packages, codes, hotfixes has been installed to the server therefore recreating a similar environment is extremely difficult and need to trace back all the historical changes or related deployment documents.
On the other side, “Declarative deployment” focuses on describing a desired stage of the system, what a system do look like at the particular version which allow us to create a system at a required stage (version) without knowing history of the deployment or whatever stage is the current system.
Let’s go back to the above example, a declarative deployment model would consist all information of packages need to be installed to make the system at production-alike stage, and for any new release all those packages information remain the same except the ones with new version.
That means, all the packages from the beginning of the project are all reflected in the declarative model.
Read more about declarative-vs-procedural.
2) How to achieve?A Traditional way for declarative deployment is using fat-package:Fat-package is a normal package which declares all dependencies sub-packages, and uses “content-package-maven-plugin” to create a combined-big-package all-in-one.
<plugin> <groupId>com.
day.
jcr.
vault</groupId> <artifactId>content-package-maven-plugin</artifactId> <extensions>true</extensions> <configuration> <filterSource>${basedir}/META-INF/vault/filter.
xml</filterSource> <verbose>true</verbose> <failOnError>true</failOnError> <group>My-Group</group> <version>${bamboo.
buildKey}-${bamboo.
buildNumber}-${project.
parent.
version}</version> <embeddeds> <embedded> <groupId>my.
custom</groupId> <artifactId>my-custom.
core</artifactId> <target>/apps/my-custom/install</target> </embedded> </embeddeds> <subPackages> <subPackage> <!–This package normally is not a part of the fat-package, I put it here to just demonstrate my point–> <groupId>adobe.
binary.
aem.
64.
servicepack</groupId> <artifactId>AEM-6.
4.
3.
0-6.
4.
3</artifactId> <filter>true</filter> </subPackage> <subPackage> <groupId>com.
adobe.
acs</groupId> <artifactId>acs-aem-commons-content</artifactId> <filter>true</filter> </subPackage> <subPackage> .
other packages.
</subPackage> </subPackages> </configuration></plugin>Fat-Package has some disadvantages, the obvious one is that it is very heavy if the application consists many packages, including the custom and third-party ones.
Furthermore, it is time consuming when installing fat-package because all the sub-packages need to be reinstalled unnecessarily every time.
A second approach is using a custom CLI (I call it AEM-CLI) which I will discuss in the rest of this article.
AEM-CLI is mainly making the declarative deployment easy, and possible to solve all the problems of fat-package approach I mentioned above.
3) The deployment architecture with AEM-CLIWhy CLI?CLI is a standalone small program and would not cause any impact to the AEM application.
In addition, it is very easy to inject the CLI to the AMI (Amazon Machine Image) or to the host machine of AEM instance (The same is true for other kinds of Machine Image).
A typical deployment process with AEM-CLI:Note that, deployment server in my case is Bamboo or Jenkins.
Or it could be a simple Shell script running in some where.
And deployment-server can communicate with CLI by SSH protocol.
There is an another way for using the CLI is putting it in deployment-server, and remotely deploy to all AEM instances, but this approach may have some performance and security issue so I do not recommend it.
** There is one more advantage of using AEM-CLI, is that it makes deploying to AEM server from scratch possible and easy.
The following chart will show the process:Notes, maybe in another article I will discuss more about Blue-Green deployment, and even real-time scaling AEM just as easy as other stateless applications, by adopting AEM-CLI and other techniques.
4) How to implement an AEM-CLI for declarative deployment:Some main features of AEM-CLI application:- First one is the ability to read a list of declared packages from a particular text file and then download them from the internet (Repository Server) to local storage.
– Secondly, the CLI needs to talk with AEM server via REST APIs or JMX in order to upload, install, replicate and do health-check to the server.
– And some extra utilities to make to CLI user friendly and flexible enough.
Now let’s go closer to the implementation details:Here, I use YAML format for declarative packages model — and name it packages-0.
0.
1.
yaml.
Or you could use Json, Xml, or whatever format you feel comfortable.
Following one is my sample:version: 0.
0.
1packages: # AEM 6.
4 Service Pack 3 – group: adobe.
binary.
aem.
64.
servicepack name: AEM-6.
4.
3.
0-6.
4.
3 version: 1.
0 type: zip # acs-aem-commons-content – group: com.
adobe.
acs packageGroup: adobe/consulting name: acs-aem-commons-content version: 3.
4.
0 type: zip # Netcentric/accesscontroltool – group: biz.
netcentric.
cq.
tools.
accesscontroltool name: accesscontroltool-package version: 2.
3.
2 type: zip # Adobe Experience Manager Core WCM Components Full Package group: com.
adobe.
cq name: core.
wcm.
components.
all version: 2.
3.
0 type: zip # Custom packages – group: my.
custom name: my-custom.
ui.
apps version: 1.
5.
1 type: zip – group: my.
custom name: my-custom.
ui.
config version: 1.
5.
1 type: zip – group: my.
custom name: my-custom.
ui.
content version: 1.
3.
0 type: zip – group: my.
other.
custom name: my-other-custom.
ui.
app version: 1.
0.
0 classifier: activation-tree type: zip # And many more more other packages.
I would use Java as the language in this article as it is the language I am most familiar and JVM is already available in AEM server so no need to install any extra dependency.
But you can use Nodejs, Python, GoLang, Ruby or even Shell-Bash script to write it.
First of all, to read an YAML file in Java:// Dependencies//'com.
fasterxml.
jackson.
dataformat:jackson-dataformat-yaml:2.
9.
8//'com.
fasterxml.
jackson.
core:jackson-databind:2.
9.
8'@Datapublic class Yaml { private String version; private List<Artifact> artifacts;}@Datapublic class Artifact { private String group; private String name; private String version; private String type; private String classifier; private boolean force;}public Yaml pasteYaml(File yamlFile) { try { ObjectMapper mapper = new ObjectMapper(new YAMLFactory()); Yaml packages = mapper.
readValue(yamlFile, Yaml.
class); return packages; } catch (Exception err) { log.
info("Pasting YAML exception: {}", err); } return null;}Then convert an artifact to a downloadable URLpublic static String getArtifactPath(String group, String artifactId, String version, String packageing, String classifier) { final StringBuffer stringBuffer = new StringBuffer(); stringBuffer.
append("/").
append(StringUtils.
replace(group, ".
", "/")); stringBuffer.
append("/").
append(artifactId); stringBuffer.
append("/").
append(version); stringBuffer.
append("/").
append(artifactId); stringBuffer.
append("-").
append(version); if (StringUtils.
isNoneEmpty(classifier)) { stringBuffer.
append("-").
append(classifier); } stringBuffer.
append(".
").
append(packageing); return stringBuffer.
toString();}// There are maybe many maven repositories, such as mavenCentral, Jmaven, or your company own mavenRepo, etc.
// So you need to check which repository does each artifact belong to before start download it.
String mavenRepositoryUrl = "https://repo.
adobe.
com/nexus/content/groups/public";String URL = mavenRepositoryUrl + getArtifactPath(.
);download(URL);After downloaded all artifacts locally, the next step is to upload and install them to AEM instance “one by one” with “health-checking step in between”.
To upload a package file to AEM//AemPathConstants.
PKG_MANAGER_JSON_PATH = "/crx/packmgr/service/.
json"@Datapublic class CrxPackageManagerResponse { private boolean success; private String msg; private String path;}public CrxPackageManagerResponse uploadPackage(AemInstance instance, File file, boolean force) { try { URI uri = URI.
create(instance.
getServer() + AemPathConstants.
PKG_MANAGER_JSON_PATH); log.
info("Sending Upload Request: {}, file: {}", uri, file.
getName()); MultiValueMap<String, Object> body = new LinkedMultiValueMap<>(); body.
add("package", new FileSystemResource(file)); body.
add("cmd", "upload"); body.
add("force", String.
valueOf(force)); // force should be false, unless special cases.
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, RequestUtils.
authenticatedHeaders(instance, MediaType.
MULTIPART_FORM_DATA)); ResponseEntity<CrxPackageManagerResponse> responseEntity = restTemplate.
postForEntity(uri, requestEntity, CrxPackageManagerResponse.
class); log.
info("Upload Response: {}", responseEntity); return responseEntity.
getStatusCode().
is2xxSuccessful()?.responseEntity.
getBody() : null; } catch (Exception er) { log.
error("Upload exception: {}", er); throw new CrxUploadException(er.
getMessage()); }}// Note:// CrxPackageManagerResponse.
path will looks like: /etc/packages/my-group/my-custom.
apps.
zip// this information will be used later to install that package.
Note: with the “force” of false, all the installed packages will not get re-upload and install, unless there is a special case we need to re-install the package every time deployment.
Thus this practice solve the problem of installing unnecessary packages in fat-package approach.
And final step, to install the previous uploaded package://fullEtcPackagePath is returned by the upload function//If fullEtcPackagePath is null, the package is no need to install.
public CrxPackageManagerResponse installPackage(AemInstance instance, String fullEtcPackagePath) { try { URI uri = URI.
create( instance.
getServer() + AemPathConstants.
PKG_MANAGER_JSON_PATH + fullEtcPackagePath ); log.
info("Sending Install Request: {}", uri); MultiValueMap<String, Object> body = new LinkedMultiValueMap<>(); body.
add("cmd", "install"); HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, RequestUtils.
authenticatedHeaders(instance, MediaType.
APPLICATION_FORM_URLENCODED)); ResponseEntity<CrxPackageManagerResponse> responseEntity = restTemplate.
exchange(uri, HttpMethod.
POST, requestEntity, CrxPackageManagerResponse.
class); log.
info("Install Response: {}", responseEntity); return responseEntity.
getStatusCode().
is2xxSuccessful()?.responseEntity.
getBody() : null; } catch (Exception er) { log.
error("Install exception: {}", er); throw new CrxInstallException(er.
getMessage()); }}Similarly, we can create a replicationPackage() function just like the installPackage().
* Note that, one of the special attitude of an OSGi application is that it sometimes needs to restart internally all of it’s bundles after be updated by a particular new bundle or when a particular configuration get updated, then during this internal restarting, the whole system is freak-out and the installation process has to wait.
This characteristic makes the installation not as simple as just posting all packages, instead of, we need to check AEM healthy status in between two installations and frequently need to pause the process in order to let the server become stable again to handle the next installation requests.
Fortunately, the AEM system provides a REST Api to expose all the OSGi bundles stages (Installed, Resolved, Active, Fragment, Restarting), so we can utilise this feature to predict whether the system is stable enough:@Datapublic class BundlesJson { private String status; /* This contains a summary of all bundles status which we need to refer*/private List<Integer> s; private List<Bundle> data;}private static final int BUNDLE_RESOLVED_ORDER = 3;private static final int BUNDLE_INSTALLED_ORDER = 4;// AemPathConstants.
OSGI_BUNDLES_LIST_JSON = "/system/console/bundles.
json"public boolean isHealthy(AemInstance instance) { try { URI uri = URI.
create(instance.
getServer() + AemPathConstants.
OSGI_BUNDLES_LIST_JSON); log.
info("Sending Request: {}", uri); HttpEntity<String> requestEntity = new HttpEntity<>(RequestUtils.
authenticatedHeaders(instance, MediaType.
APPLICATION_JSON_UTF8)); ResponseEntity<BundlesJson> responseEntity = restTemplate.
exchange(uri, HttpMethod.
GET, requestEntity, BundlesJson.
class); log.
info("Response: {}", responseEntity); return responseEntity.
getStatusCode().
is2xxSuccessful() // Check if no RESOLVED bundle && responseEntity.
getBody().
getS().
get(BUNDLE_RESOLVED_ORDER) == 0// Check if no INSTALLED bundle && responseEntity.
getBody().
getS().
get(BUNDLE_INSTALLED_ORDER) == 0; } catch (Exception er) { log.
error("uploadAndInstall exception: {}", er); } return false;}public void waitForAemStable(AemInstance instance, int timeoutInSecond) { int totalWaitedTime = 0; while (!isHealthy(instance)) { try { TimeUnit.
SECONDS.
sleep(DURATION_RETRY_IN_SECOND); totalWaitedTime += DURATION_RETRY_IN_SECOND; } catch (InterruptedException e) { log.
info("waitForAemStable->InterruptedException: {}", e); } if (totalWaitedTime > timeoutInSecond) { log.
error("Aem is remaining unhealthy after: {} (seconds)", timeoutInSecond); // It is possible to let aem-cli restart the AEM in this casethrow new AemUnhealthyException("Aem is remaining unhealthy after: " + timeoutInSecond + " (seconds)"); } }}So the whole installation process looks like:readYamlFile()packagesFiles = downloadAllArtifacts();forEach(file : packagesFiles) { String etcPath = uploadPackage(file); if etcPath not empty: installPackage(etcPath); waitForAemStable();}The last piece of codes is how to create a command line interface in Java, and I would recommend to use picocli library.
Furthermore, if you want to utilise the dependencies injection and fat-jar packaging, you can use Spring Framework, and Spring-boot as well.
// Spring boot, CommandLineRunner@SpringBootApplication@Log4j2public class AemCliApplication implements CommandLineRunner { // pococli code@Autowiredprivate CommandLine commandLine; public static void main(String[] args) { SpringApplication.
run(AemCliApplication.
class, args); } @Overridepublic void run(String.
args) { commandLine.
parseWithHandlers(new CommandLine.
RunAll().
andExit(0), CommandLine.
defaultExceptionHandler().
andExit(1), args); }}And a command in pococli would look like:@Component@Log4j2@Getter@ToString@CommandLine.
Command(footer = "Copyright(c) 2019", mixinStandardHelpOptions = true, versionProvider = CliVersionProvider.
class, subcommands = {CommandLine.
HelpCommand.
class})public class AemMainCommand implements Runnable { @Option(names = {"-n", "–name"}, description = "AEM instance name.
Example SIT, UAT, PROD") private String name; @Option(names = {"-s", "–server"}, description = "AEM Server including protocol and port.
Ex: http://localhost:4502") private String server; @Option(names = {"-u", "–user"}, description = "User") private String user; @Option(names = {"-p", "–pass"}, description = "Password") private String password; @Option(names = {"-t", "–type"}, description = "AEM Instance Types: ${COMPLETION-CANDIDATES}") private InstanceType type; @Autowired@ToString.
Exclude @Getter(AccessLevel.
NONE) private DefaultCliConfig defaultCliConfig; @Autowired@ToString.
Exclude @Getter(AccessLevel.
NONE) private AemInstallService installService; @Overridepublic void run() { log.
info("Aem->: {}-{}", this::getAemInstance); installService.
install(getAemInstance(), .
); } public AemInstance getAemInstance() { AemInstance defaultInstance = defaultCliConfig.
getAem(); return new AemInstance( StringUtils.
isNoneEmpty(name)?.name : defaultInstance.
getName(), StringUtils.
isNoneEmpty(server)?.server : defaultInstance.
getServer(), type != null?.type : defaultInstance.
getType(), StringUtils.
isNoneEmpty(user)?.user : defaultCliConfig.
getAem().
getUser(), StringUtils.
isNoneEmpty(name)?.password : defaultCliConfig.
getAem().
getPassword() ); } public Integer getCheckHealthTimeout() { return defaultCliConfig.
getCheckHealthTimeout(); }5) AEM-CLI expected usage:Assuming we all finished those above implementation steps.
The usage of AEM-CLI would look like:// Using the AEM-CLI is just simple as any other typical CLIs:$ java -jar aem-cli.
jar install /tmp/packages-0.
0.
1.
yaml// Or we can even wrap it in a SH script and use like:$ aem-cli install https://mynexus.
com/release/packages-0.
0.
1.
yaml// Others functions:$ aem-cli –help$ aem-cli config –server http://localhost:4502 –user admin .
$ aem-cli report –lastestFrom now on, instead of deploying a list of binary artifacts to AEM servers, we just need to release the package-xxx.
yaml and AEM-CLI will take care the rest of the process.
6) Conclusion:This article shows how to implement a custom CLI in order to improve the deployment process, especially following “declarative deployment” approach to make the re-creation/redeployment just as simple as possible.
If you find this kind of concept is interesting and helpful please leave your comment, or any suggestion so I can consider to start a new open source project of this one.
Also, I hope AEM 6.
5 will officially support the concept of “composite-nodestore”, if then it will be more straightforward to deploy AEM follow blue-green approach or even more.
Here is the diagram to demonstrate the point:The inspiration for this article comes from:https://github.
com/Cognifide/gradle-aem-pluginhttps://blog.
gruntwork.
io/why-we-use-terraform-and-not-chef-puppet-ansible-saltstack-or-cloudformation-7989dad2865chttps://devops.
com/perfect-combination-imperative-orchestration-declarative-automation/https://github.
com/bjorns/aem-cmdhttps://adapt.
to/2017/en/schedule/zero-downtime-deployments-for-the-sling-based-apps-using-docker.
htmlhttp://www.
jetteroheller.
com/blue-green-deployment-on-adobe-experience-manager/.