Useful DOS commands

From the 80s to 2020, I find myself living more and more in DOS, Windows, Powershell, Bash/Zsh, Kube and Cloud shells, and looking forward to leveraging the new Windows Terminal once my work laptop gets upgraded.

I appreciate the complexity and flexibility that Zsh has vs. DOS, and miss a lot of the features.

Sometimes though, there are some interesting tricks I forget about when using DOS/Windows/Powershell.

Grab the history to a file.  I always hit F7, and would appreciate having my session's file around for a basic command window.

doskey /HISTORY > history.txt
https://www.howtogeek.com/298163/how-to-use-your-command-history-in-the-windows-command-prompt/

Also see Start Transcript
https://4sysops.com/archives/powershell-transcript-record-a-session-to-a-text-file/

Pipe a python script's output and stderr output
python -m myscript.py &2>1

Teeing Objects
https://adamtheautomator.com/tee-object-powershell/

Ascii Doom
https://dario-zubovic.itch.io/1337-doom

Last one should be cross-platform!

Common Kubernetes Commands

The Kubernetes Client kubectl recently became an independent tool outside of the main Kubernetes repo.

Some common Kubernetes kubectl commands.

https://kubernetes.io/docs/reference/kubectl/overview/
https://github.com/kubernetes/kubectl/
https://kubernetes.io/docs/reference/kubectl/cheatsheet/

Get the current cluster / namespace context

kubectl config get-context

Use the specified cluster context
kubectl config use-context my-cluster-name

Which cluster am I on
kubectl cluster-info

Create my service/deployment/secret/configmap/replicationcontroller/pod/job
kubectl apply -f myyaml.yaml

Delete the applied object(s)
kubectl delete -f myyaml.yaml

Delete a pod
kubectl delete -n dev mydevpod

Get a list of pods for all namespaces
kubectl get pods -A

Get the logs for a pod
kubectl logs -n dev mydevpod-abcde

kubectl get internals
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/get/get.go

Copy files to a pod
kubectl cp /tmp/foo_dir <some-pod>:/tmp/bar_dir

Other commands
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/cmd.go

get, describe, delete, label, edit, patch, annotate, expose, scale, autoscale, taint, rollout, logs, exec, rolling-update, cordon, config use-context, config delete-cluster, cp

kubectl get namespace
kubectl config get-context
kubectl config get-clusters
kubectl config get-users

Microsoft Learning Resources and Restructuring Big Data

Microsoft has recently disrupted their learning paths and retired a number of exams.  I am happy to have passed AZ-102 - Microsoft Azure Admin Certification Transition last year, while it was in beta. 

A number of exams have been retired recently or are set to retire.
https://buildazure.com/2018/12/21/microsoft-exams-scheduled-to-retire-in-early-2019/

Microsoft is leveraging edX for its Microsoft Professional and Capstone tracks.  Pick a topic and go.

Microsoft Virtual Academy is going away and being replaced with Microsoft Learn.  What's great about learn is that it provides a no-cost sandbox in Azure for learning, directly in the labs!  Microsoft is pursuing developer adoption, and developers have concerns with their credit cards and cloud services.  A former colleague of mine rang up a large bill in Amazon after leaving on some Hadoop clusters he was using for training.  This fear of costs and the pay-as-you-go model will continue to limit the cloud experience for entry-level developers.

Speaking of cost containment, it is 2019 and we still can't pause HDInsight clusters.... I think that this feature alone would "Spark" increased growth of Hadoop on Azure.  Azure Databricks and its job-based cluster model seems to have the right approach, and the friction of spinning up and pausing clusters is much less than in HDInsight. 

With Databricks being the fastest growing company in North America, I see adoption of Cloudera (+ the former Hortonworks) and HDInsight in the cloud continuing to increase.  The vast zoo of services between Hortonworks Data Platform (HDP) and Cloudera Data Platform (CDP) being combined and deprecated will make for an interesting gene pool.

And all will sit on Azure Data Lake Gen2 Blob Storage?  Microsoft marketing needs a better name than hierarchial namespace for their game-changing big-data filesystem....

When Apache and Microsoft worlds collide - there are some amazing code reviews coming out of those JIRA tickets where companies such as Microsoft enter the open-source arena.

Microsoft Azure HDInsight Big Data Analyst

Azure CosmosDB Resources

A few links to assist with CosmosDB

CosmosDB Indexes

http://muratbuffalo.blogspot.com/2018/08/logical-index-organization-in-cosmos-db.html?utm_content=78727690&utm_medium=social&utm_source=twitter

CosmosDB Cheat Sheets
https://docs.microsoft.com/en-us/azure/cosmos-db/query-cheat-sheet?WT.mc_id=twitter&utm_content=78509449&utm_medium=social&utm_source=twitter

CosmosDB Like A Pro
https://medium.com/@david.nissimoff/azure-cosmos-db-like-a-pro-minimize-cost-maximize-productivity-24dfee205c0

CosmosDB Ignite 2018 Sessions
https://www.youtube.com/playlist?list=PLmamF3YkHLoLkL4Q-fRgbG8URj503ZMgd

Durable Functions goes GA

Azure Durable Functions is an extension of Azure Functions and Azure WebJobs for managing state, checkpoints, and restarts.

• They can checkpoint on await and maintain state
• They can orchestrate other functions
• They define workflows as code. 

There are some complex problems that can be solved by leveraging durable functions, including processes requiring manual intervention.

Be sure to view the release notes and open issues to make sure durable extensions will work in your scenario.

https://github.com/Azure/azure-functions-durable-extension/releases/tag/v1.4.0https://github.com/Azure/azure-functions-durable-extension/issues

 

Azure DocumentDB Emulator Released

Azure DocumentDB Emulator was released late last year.  It provides an on-premise way of testing DocumentDB databases without spinning up a DocumentDB service in Azure.  Based on the comments thread it is a bit unstable, yet there is still plenty of interest.

Most enterprise organizations and developers doing testing against cloud services require on-premise versions of those services.  This simplifies development, reduces latency instead of connecting to cloud resources, and reduces costs. For customers using on-premise builds, it is vital to making sure that all builds and tests are contained in their own silo.

https://docs.microsoft.com/en-us/azure/documentdb/documentdb-nosql-local-emulator

Comparison of HBase, MongoDB and DocumentDB
http://db-engines.com/en/system/HBase%3BMicrosoft+Azure+DocumentDB%3BMongoDB

DocumentDB provides protocol support for any apps written to use MongoDB to convert MongoDB servers to a PaaS Azure model.  Some features such as aggregation pipelines are not yet available.
https://docs.microsoft.com/en-us/azure/documentdb/documentdb-protocol-mongodb

DocumentDB has been around since 2014, however MongoDB is still the beast in its category.
http://db-engines.com/en/ranking/document+store

An intro to DocumentDB
https://docs.microsoft.com/en-us/azure/documentdb/documentdb-introduction

Analyze data with HDInsight and DocumentDB
https://docs.microsoft.com/en-us/azure/documentdb/documentdb-run-hadoop-with-hdinsight

The MongoDB Spark Connector
https://github.com/mongodb/mongo-spark

DocumentDB as a device registration store for Azure IoT Hub
https://docs.microsoft.com/en-us/azure/iot-suite/iot-suite-what-are-preconfigured-solutions

I'm still looking for a good way to do on-premise Azure Data Factory tests for simulating pipelines.... just upvoted Support MSBuild...

https://feedback.azure.com/forums/270578-azure-data-factory?filter=new&page=4

Awesome Awesomeness Lists including Cloud, IoT and Mobile

A few resources for mobile devices, REST APIs and other awesome awesomeness lists.

Awesome Mobile UI/UX Interfaces
https://github.com/cymcsg/Awesome-Mobile-UI

Awesome API
https://github.com/Kikobeats/awesome-api

Awesome IoT
https://github.com/HQarroum/awesome-iot

Simulate Popular Mobile Devices for Testing
https://www.asp.net/mobile/device-simulators

Mobile Dev Learning
https://github.com/MakinGiants/awesome-android-learner

Awesome Awesomeness List
https://github.com/bayandin/awesome-awesomeness

Curated List of Awesome Lists
https://github.com/sindresorhus/awesome

The Viral Progressive Web App
https://cloudfour.com/thinks/the-business-case-for-progressive-web-apps/

Progressive Apps
https://infrequently.org/2015/06/progressive-apps-escaping-tabs-without-losing-our-soul/

Service Workers
https://developers.google.com/web/fundamentals/getting-started/primers/service-workers

Getting Started with WebVR
https://developers.google.com/web/fundamentals/vr/getting-started-with-webvr/