Kinesis Analytics now gives you the option to preprocess your data with AWS Lambda. This gives you a great deal of flexibility in defining what data gets analyzed by your Kinesis Analytics application. You can also define how that data is structured before it is queried by your SQL.
It continuously reads data from your Kinesis stream or Kinesis Firehose delivery stream. For each batch of records that it retrieves, the Lambda processor subsystem manages how each batch gets passed to your Lambda function. Your function receives a list of records as input. Within your function, you iterate through the list and apply your business logic to accomplish your preprocessing requirements (such as data transformation)
The input model to your preprocessing function varies slightly, depending on whether the data was received from a stream or delivery stream
Last week, an interesting industry news that Microsoft and Amazon announced a surprise partnership to build AI platform (named Gluon) for an enterprise. Gluon makes it easier for developers to build AI/machine learning systems, and related Apps with open source concept.
I've an interesting dimension of this technology partnership to challenge Google's big area of AI dominance using Tensorflow.
Google TensorflowGoogle already has a head start with a tool called Tensorflow, which is free and open source and aimed at helping developers build machine learning apps. Tensorflow is immensely popular with developers.
In fact, it's the the fifth most popular project (by stars) on GitHub out of the over 2 million hosted on that site where open source projects are shared. Quick introduction video is shown at https://www.youtube.com/watch?v=MotG3XI2qSs
Strategic Partnership Machine learning and AI are the next big things in cloud computing, with the potential to cause significant changes to the cloud business that Amazon and Microsoft have long dominated. Microsoft and Amazon have been known to cuddle up on other AI types of tech.
In August, the two announced they were partnering to make their two voice assistants work better together , Amazon Alexa and Microsoft Cortana.
Joint Venture GluonMicrosoft and Amazon have joined forces to help spread artificial intelligence across apps. They released a new tool for developers called Gluon as a free and open source project, meaning anyone can use it or work on it and contribute to it for free.
Gluon's role is to add a layer that makes MXNet and CNTK easier to use, work with and program. Only the MXNet version was released now; but the CNTX version of Gluon is promised to come soon. Short introduction is shown at https://www.youtube.com/watch?v=NxAYuw5QQ_8
Ease for AI DevelopmentIn any case, the competition to create more AI tools for developers, and make them easier to use. Demand of Artificial Intelligence in various industries, are reflected in 3 years scorecard
Industry Graph share is led by Neo4j and Titan, the latter recently acquired by DataStax (DataStax Enterprise Graph).
Social graphs are a prime example of utilizing the graph model, Dr. Xu (PhD from UCSD) was working at Twitter till 2011, and the graph databases that were around at the time could not cope.
He has 26 patents in distributed systems & databases, led Teradata's big data initiatives, and worked on Twitter's distributed data infrastructure. So when faced with that problem, Xu saw an opportunity and went off to create a solution.
Xu founded GraphSQL in 2012 and has been working with a team of 30 engineers since. Today GraphSQL is officially entering a new stage in its development, including a new name: TigerGraph.
The product is now generally available, a series A founding round of US$33 million is announced and a hosted version of TigerGraph based on Amazon EC2 is launching.
TigerGraph also supports different graph partitioning algorithms enabling it to split very large graphs over a distributed architecture. This can be done either automatically, or as specified by users using application-specific partitioning strategies.
There probably is a hefty price tag that goes with TigerGraph, but for the ones that can afford it, it looks like it can deliver some substantial benefits.
1. Nullable Reference Types
Consider a scenario where you know that the nullable variable x isn’t actually null, but you can’t prove that to the compiler. In this case you can use x!.Method() to suppress the compiler warning about potential null reference exceptions.
2. Extension Everything
As with interfaces, you cannot define instance fields in extensions but you can simulate them using ConditionalWeakTable. You can also define static fields.
3. Default Interface Implementations
The primary benefit of default interface implementations is that you may be able to add new methods to an existing interface without breaking backwards compatibility.
4. Async Streams (a.k.a. foreach async)
This is referred to as a “pull model”. By contrast, IObservable is a “push model”, which means the producer can flood the consumer with a higher flow rate than it can handle.
5. Extension Interfaces
Extension interfaces, the ability to add new interfaces to existing classes, is also being considered.
My closing note is C# is ahead of tech capabilities and roadmap, on comparison with Java.
.NET Core 2.0 is also released today. This is the second major version of .NET Core and this release focuses on performance improvements and expanding the set of APIs available via .NET Standard 2.0. It includes the runtime and libraries for .NET Core as well as the tools for building, debugging and running .NET Core applications.
Visual Studio for Mac version 7.1 is also available today. It adds support for .NET Core 2.0 targeting in console apps, web apps, and web services. It also enables creating .NET Standard 2.0 in library projects, to share more code across projects. Like Visual Studio 2017, a lot of the improvements in this update center on reliability.