Browsing Category

Digital transformation

Digital transformation, IT is business, Practical examples

Using Machine Learning in real life part 2/2

April 5, 2018

Now that we understood the difference about the three approaches we can have over AI with the last article, let’s dive in into the mid-term approach, always keeping in mind the borders. Given the explanation of the differences between Machine Learning approaches: a) ready-to-use APIs, b) training a model, and c) creating a model, let’s talk about training (using) a model.

Training a model

This is the mid-term approach to AI (Machine Learning) problems. Once you found out your problem can’t be solved by any ready-to-use API, try this approach. Just because there is no ready-to-use API, it doesn’t mean nobody ever tried to solve your problem generically and widely speaking. There is a high probability that your problem already can be solved by an existing model. Using this approach you will have to look for three things. It’s not a dependency, since the third step can be left away in some cases (example below), but they are a sequence.

Finding the best model

This is the part where you will have to have someone with good experience in this subject. There are different models to solve the same problem, as an instance. And there are also many problems without models covering it. You will have to find the best model that fits best to your needs. You will have to check the trust percentage that model gives, if it checks all the information you have, if you will have to adapt any information you already have to use that model, and many other things;

We can split the models into three different groups:

  • Models for supervised training

    It happens when you have the information that the algorithm must reach conclusion X (objective) after evaluating A (info 1), B (info 2) and C (info 3) information; Example: you know that sneezing (info 1), high body temperature (info 2) and pain over all the body (info 3) means you have the flu (objective). Here I present some well-known models:

o Linear regression – https://docs.aws.amazon.com/machine-learning/latest/dg/types-of-ml-models.html – It is good to work with number prediction. Examples: What will be the weather for tomorrow? For how much this house will be sold?

o Decision tree – https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.modeler.help/nodes_treebuilding.htm – Find out the disease: are the sympthoms A, B and C true? Then disease X; Are the sumptoms A, B and D true? Then disease Y;

o Bayesian network – https://pt.slideshare.net/GiladBarkan/bayesian-belief-networks-for-dummies – When we have an evidence and want to reach its cause. Belief propagation. The same health scenario above can be applied, but in an inverse way. I have the flu. I must identify in this patient we have all the sympthoms, or even without showing sneezing, he still has the flu.

  • Models for unsupervised training

    It happens when you don’t have the conclusion the algorithm must reach. You will have to check it every time it runs. Example: if the customer bought product X and Y, he may be interested in product Z. You won’t know if it is true, because the customer may get interested, but even with that, don’t buy the product. Here I present some well-known models:

Association – https://en.wikipedia.org/wiki/Association_rule_learning – Same example above of suggesting things to be bought;

o Anomaly detection – Any chart control or information where anomalies have to be alerted. Stock market or the temperature inside a factory’s chamber, as instances;

  • Models for semi supervised training

When sometimes you know what you will reach. Your problem will set if you will be able to use this model. More models can be found at https://en.wikipedia.org/wiki/Outline_of_machine_learning#Machine_learning_algorithms

Configuring a model

It happens when you have a known model and must configure it. Sometimes you won’t have to train the model.

An example for linear regression, which came from another customer: he wanted to mix many different information from many different sources and reach out how it would affect their product pricing. For each of their products, you configure the algorithm in order to understand that supply A affects 10% of final product pricing, supply B affects 50%, and etc. Knowing that, the algorithm would be able to “predict” changes on their prices and warn them to buy more or less of each supply. This way they would be in front of their competitors, saving money at the right time;

And then… Training a model

Once you have a problem requiring a model to be trained to identify your target, you will have to have data to train your model. The image analysis that cloud providers provide through APIs are great examples. Once you upload an Eiffel Tower image there, the algorithm already knows there is an Eiffel Tower within your image. But how do they do that? They have already trained the model to understand patterns on the image and then classify it. It’s the same thing that Facebook does every time it recognizes faces on your uploaded photos. For the Facebook example, it gets even more impressive because Facebook trains their algorithm with everybody’s faces. Then they know that your image has a photo of your specific friend, and suggest you to tag that guy! It’s not just a generic person recognition like other models do.

 

How to do that?

At last, there are many tools, such as Google AutoML, Amazon Machine Learning, Watson and TensorFlow (open source tool). The providers solutions allow you to send a given model to cloud and then use their infrastructure to run, train and consume it;

Digital transformation, IT is business, Practical examples

Using Machine Learning in real life part 1/2

March 30, 2018

Everybody has been talking about Machine Learning, and everybody wants to get benefits of Artificial Intelligence. It is a new thing IT managers grabbing that old problem from inside the old locker and thinking: “hey! Maybe new Watson can solve it for me!”. But every time I hear someone new asking about how to solve a problem with AI, the problem looks like something never seen before. Every day a new solution is researched to a new problem. If every day a new will comes up, how can we identify what are the borders for AI? Since AI stands for “Artificial Intelligence” what is the “intelligence” border? What can and what cannot be solved with what we have today?

How to identify how hard it will be to find a AI to your scenario

Machine Learning projects can be split into three groups:

  1. Using a ready-to-use open API (this first article focuses here)
    • What is it? This is the fastest approach. There are a lot of APIs ready to be accessed and to be added to your solution. There are a lot of benefits, and you just have to pay for that. There is a table and more details below;
    • How long will it take? You can get results from some tests within one day;
    • Some benefits:
      • a) They are ready to use! You just have to plug them to your app. Anyone can do that;
      • b) Their suppliers will keep training the model as you go! So, it won’t ever be outdated;
      • c) The competition between suppliers will grant you always well trained models and non-stop improvements and updates;
      • The items above will be very expensive to reach when outside this approach;
    • Some restrictions:
      • a) It doesn’t belong to you. It means you can’t change anything on how it works. It’s just you asking: “hey, please classify this image!”. Then the answer would be: “cool! Your image has a woman on it”. But you can’t ask back: “what’s the woman’s hair color?”;
    • Conclusions?
      • If the open APIs fit your needs, don’t ask again and start using them by now! Don’t worry about suppliers grabbing your information or whatever like that. Once you pay for the tool, you have a contract where they say they won’t use your data. It’s the same thing of cloud;
      • If it doesn’t fit exactly your needs, try to understand how important is to have that 1% more of trust over that AI judgement. If you really need something more precise, jump to “training a model”;
  1. Training a model
    • Mid-term approach. There are a lot of different models ready to be added to a project, configured, trained and then used. I will talk about this approach in the next article;
  2. Building a model
    • Keep in mind it won’t be an IT project for a while. You will have to have people from physics, mathematics and specialists on your business, and really good information to add to your project in the very beginning. Once they finish the model (it can take up to 2 years. Maybe more), then it will turn into a regular IT project starting from “training a model”. Both of this projects (building and training) will take for sure more than 2 years of research and testing. If it is a key process to your company, don’t waste one more second and start this project. The sooner you start, the sooner you will get the benefits;

 

Cool! What are the ready-to-use APIs?

My suggestions are inside this table below. But the options are not limited to it. You can find many others. They are maintained by the biggest cloud and AI players. It means you can trust it, and probably for what they focus, they are the best you will find.

Feature Google IBM Microsoft Amazon
Chatbot related DialogFlow Watson Assistant and Virtual Agent Bot Lex
Video Analysis Video Intelligence Intelligent Video Analytics Video Indexer Rekognition
Image Analysis Vision Visual Recognition Computer Vision API Rekognition
Speech to Text Speech Speech to Text Bing Speech Transcribe
Text to Speech Text to Speech Bing Speech Text to Speech
Natural Language Classifier Natural Language Natural Language ClassifierNatural Language UnderstandingPersonality Insights and Tone Analyzer Language Understanding Comprehend
Translation Translate Translator Translator Translate
Trends search and analysis Trends Discovery and IBM’s Discovery News
Find patterns over unstructured text Knowledge Studio
Content moderator* Anomaly Detection
Jobs discovery Job discovery**

* Google, IBM and Amazon have content moderator built-in their products. Microsoft has this specific product looking for anomalies only.

** Job Discovery is a private tool available for only few partners.

 

Two examples to talk about the borders again
Image recognition

Just like the example above: a customer came to me asking about a solution to identify people on images. Great! Let’s use Google’s Vision! Vision identifies people on photos and gives a lot more information about the colors on that image, about places that image may contain, and etc. But then the customer asked me: I want to recognize if it is a woman. I said ok! And then: I want to recognize the woman’s hair color. Ok, all open APIs are off the game. Let’s find a model, train it and then get hair colors. For you to be able to answer those questions there is no shortcut. You will have to read the documentation of each open API you find and run tests on it.

Language defect recognition

Another customer came to me asking if they could give a microphone to their employees in order they could operate a system just giving voice commands. Ok, it’s not new. We could use a mix of speech-to-text and natural language processing APIs, let’s move ahead! But then the customer said the system should recognize internal terms like acronyms and words they invented to communicate with each other. Erm… it’s not possible. You can’t train ready-to-use APIs to understand your very own specific terms. The easiest way there was to suggest the operators to change the words for some others the system would recognize. Otherwise they would have to grab models, configure and train them to understand the new words.

 

Then, why don’t you give your first AI step over ready-to-use APIs evaluation? The sooner you start, the sooner you will understand how to approach that old problem.

Digital transformation, IT is business, Practical examples

Cloud is the new black

March 15, 2018

I heard “cloud is the new black” during a training session inside a Google’s office. You can guess it means that cloud is basic. But why did they say it? Google didn’t forge this beautiful sentence. Gartner did.

What Gartner means by “cloud is the new black”? In short terms, Gartner said the cloud Market is a US$ 1 trillion Market. Right now it’s just US$ 56 billion.

Google repeats that for comercial purposes of course. The reasons are the same Microsoft, AWS and any other cloud provider do: scalability, stability, abstraction of infrastructure processes, and the same bla bla bla. And I totally agree with their reasons. But without considering this technical questions, to the final business results $$$, why is the cloud so basic?

 

How does that come to our business?

Yes, there are already a lot of companies moving to cloud and starting their applications inside the cloud. But I still can easily find many companies still not even considering the cloud move. Inside my reality it’s hard to understand. How can they not see cloud benefits? How can they still use their machines and spend millions of dollars buying more and more storage every 6 months? For me it’s waste of time. I’ll explain why.

Physical space

The rooms where the machines are hosted. They cost Money. For few companies, I’ve already seen very expensive entire blocks inside noble areas like São Paulo being used to host… machines. They don’t need the datacenter to be that close to the offices. The latency doesn’t matter that much. I’m 100% sure. For sure if they remove everything there, and rent the space, the renting revenues will pay for a big slice of cloud cost monthly. What if they sell it? It would mean investments for areas that are needing that Money to innovate and be in front of their competitors. Because of that lack of money, their areas are wasting time. It’s waste of time.

Rework rate

Recently a datacenter, close to the company where I work had a fire issue. Many governmental and private companies, core and non-core systems, were affected. And where were the backups? Inside the same building. Because of the fire, the fireman and police didn’t allow the technical team to get there and move the information to another datacenter. The replication wasn’t automatized. It caused more than 12h of unavailable systems. Can you imagine any company inside any industry without receiving transactions during 12h? Imagine the financial area. Hard. Now imagine a factory without systems for 12h. They won’t sell for a whole day? You could answer “Yeah, but we can ‘take notes’”. The employees don’t remember how to hold a pen this far. They also won’t know how much they produced of what they produce. They won’t know how much they spent producing things. But the main thing here is the overhead that will be created inside those companies to put everything back to systems. Talking about Brazil they can even get a ticket from government because of some missing transactions on that day. What all that means? Waste of time.

Why not cloud?

I have a client who runs a retail solution on cloud. The solution has been running for the last 2 years without downtime. Is it a core solution to their business? No, it’s not. But the fact that they don’t have headaches with that small system saves them time to think on other things. Saves tickets on traditional infrastructure teams. Saves their mental health also. Of course the cloud itself is not the only answer for the application stability. They do care about quality on their development process. Then all these benefits come easily.

 

When will companies decide to migrate to cloud?

It all makes me think that using cloud is related to maturity. Now a quick link with internet-related startups: companies who grow unbelievable percentages every year in the entire world. The biggest part of them has the cloud in common. Most of their business models wouldn’t be possible without the cloud.

The traditional companies, which already felt startups “bothering” their market shares, are moving, or already moved to cloud.

Why does that happen? Because the cloud gives them the speed they need. Things I’ve already seen in on-premise VS cloud environments:

  • A new environment to create a new app can take up to 1 month to be released by the infrastructure team to the development team to start work. It’s one month less on that project. Within cloud it’s solved in less than an hour.
  • Analytics information being generated only with the data considering the day before the current. In cloud you can have live information to take your decisions.
  • Analyzing petabytes of data without having to do that on the weekend, when there’s no concurrency with other systems running. In cloud you can do that whenever you want without buying millions of dollars of infrastructure in advance.

All these examples want to say the same thing: when the companies start feeling they are being left behind because they are slower than their competitors (either it is a startup or not), they will change.

 

So, why cloud is the new black?

So… Getting back, why cloud is the new black? Because it means saving time. Because if this text made you remember of any issue you are having, or you may have inside your company, it means you will run after it to solve. It won’t be a short run to find all the responsible for everything and asking them to change to your new conceptions. It will take weeks. Months at least. Those weeks or months spent by the team looking to fix or prevent something to happen, means weeks or months not looking to improve the business, not looking to be in front of competitors. The IT area is not the support anymore. It can’t JUST be prepared to whatever the other areas will demand. It HAS to be the one of the leading business areas. And why that is true? The IT guys know what technology can do. The other areas don’t.

Digital transformation, IT is business

It’s time to take a look at Machine Learning

February 28, 2018

Machine Learning (ML) is the word for the moment. Just like SOA has already been, just like DevOps has already been. Now everybody talk about ML. Everybody want to get its miraculous benefits but just few of them all really know what it is, its possibilities, and the shortest path to start getting its results and leveraging with it.

What is ML?

Wikipedia says “Machine learning is a field of computer science that gives computer systems the ability to “learn””.

Ok, got that. But what is Machine Learning? What does it can achieve? Let’s get to some examples:

Netflix recommendations

Netflix told, during Cassandra summit on 2016, that everything it shows are recommendations. The slides are available here (there’s also a Youtube video at the end). And people like it! 80% of what we watch are recommended. We don’t actively look for it. It’s one click far.

They got 5.2 million new users from may to june in 2017. If you were on of them, you didn’t fill a long question set giving them information about which genders you like the most, not even what you don’t. Also, they don’t have the largest number of interns in the world to read 5.2 new user’s data and humanly suggest movies for them to watch. Those recommendations were made and keep always being made and updated within few minutes after you show any kind of sign you like something. Watched something? Flagged as “watch later”? Flagged as “like” or “dislike”? It all counts. It all updates your profile and new suggestions may come.

Google Maps

Now take a look at this second example. This Google’s Street View image shows a parking lot in Primeiro de Março Street, at São Leopoldo (south Brazil small city):

I’m 100% sure the parking lot named “estacionamento beto” didn’t send their data to Google Maps team. They didn’t fill a form telling they are at São Leopoldo. Their owner probably have a mobile phone with internet access. But probably they didn’t even know it is possible, and maybe they didn’t even know what is Google Maps. But they are there:

The Machine Learning here is something already touchable using a Google API to analyze images. They analyzed the Street View image, and the machine knew it was something with commercial purposes. Then the algorithm automatically “registered” it as something to be shown in the map. If you look at it at Street View, you will see that this street block is almost entirely residential. Google’s algorithms knew what’s the different between residential buildings and commercial buildings.

Few others
  • Facebook suggests you to tag friends at their faces when you upload a new photo. It’s Machine Learning. Facebook doesn’t have one picture of each people in world to compare when they receive new pictures;
  • Almost all e-commerces we know nowadays do that, but Amazon started: products suggestion based on the last purchases you done;
  • Getting information from your bank account straight on Facebook’s chat. It can recognize many different questions for “whats my balance?”

 

How to benefit?

It’s very easy to think on new scenarios, more or less complex. Automatizing predictions or anything that today requires a human mind, is, or will be, possible.

Example! We know our stock market has patterns on its behavior and it’s affected by many things. Today many people spend their whole day (or life) analyzing data and giving predictions to others about what to do with their stocks (buy/sell orders).

The red dots (mine) on data above (from tradingeconomis.com) show moments when the market started cycles of loss over the last 10 years. During all of these examples we already had a set of technical analysis that helped us to identify when it’s more or less likely to happen again. But it (still) takes a lot of people distributed across many companies to MAYBE find something significant out. Having a good Machine Learning model can change this entire market. It can change into something unfair to put money, since parts of the analysis MAY become predictable.

 

But how to start?

A good new Machine Learning model invariably goes through a huge data quantity. The data is required to train the model and then achieve the real business goal. A good start is to set up a structured base of data to be used. You will also need a data scientist. Don’t worry, this scientist is not an alien. They are potentially very easy to find. But it’s subject for a new post.

Customers, Digital transformation, IT is business, Practical examples

Software buying process on IT evolution

February 14, 2018

The more different views you have about the same subject, the more information you will have to take your conclusions and make decisions if needed. This first article about IT evolution had a sight from developers and a very superficial business view. This new article aims to look from the perspective of clients buying software and how they are different even inside the same industries.

 

How software used to be bought

In past years, companies used to buy software like any other commodity: I want 5 cars with 4 wheels each, manual transmission and white painted. They didn’t mind on how their software is produced, or any kind of good practices to apply on the process. Yes, inside the software requirement documents were sections regarding security, protocols and things like that, but at the end of everything, the lowest price would win.

 

Old (not too much) decision matrix

The decision matrix (example below using a car buying process) were very simple when selecting the criteria. They were also superficial. The high level criteria were easily like: security levels, performance, proposed schedule, price, scope coverage, success cases, knowledge on selected technology, etc. With this decision matrix, the weights are given according to each project. But the price and schedule, after all the criteria evaluation would still determine who wins.

On the example above: what exactly is comfort and styling for this buyer? Safety can be evaluated using governamental public data. This is a parallel to show how none of the criteria is detailed.

Anyone saying that could cover the entire scope, under the asked schedule having a fixed amount of money could win the fight for the projects. If they didn’t have knowledge enough, it wouldn’t make huge difference. After everything, software was treated like any other product. It doesn’t matter how it’s made. It just matter that it works.

 

Recent decision matrix

Leaving behind government industry, and few others not affected even a bit by digital transformation, the reality has been changing.

What I’ve seen lately is an increasing spend of time on detailing the criteria. For each of the criteria, the buyers want to know what the supplier have already done, how they plan to conduct the solution and also want to evaluate alternatives for everything.

  • Security: ensure the information exchange will occur under a controlled environment? Tactics for code obfuscation. Platform natural security concerns. Cloud provider certifications, etc;
  • Schedule: from “a detailed schedule” (which is never achieved) to “a macro view of time and a mix of agile methodologies to be followed”;
  • Success cases: show where/when similar solutions were already created;
  • UX:  from “the system has to have good usability” (completely subjective) to “it must follow UX guidelines from Google and be conducted by a specific professional, not by developers”;
  • Performance: from “the system has to load under 10 seconds” to “each endpoint must answer under 2 seconds”;
  • Scope: from “do everything desired” to “let’s do the most we can inside the schedule”;

The companies want to know how their project will be conducted in details, and they also want to put their opinion on it. Since the digital transformation is making companies to turn their core into IT, the IT is not a support anymore. So now they are able to suggest things and to talk at the same level of the consultancies.

With this comparision and evolution scene, the MVP mindset is very clear. Also the goals to achieve faster time-to-market and faster revenues.

 

Different requests inside the same industry

The report above is true for 100% of companies on the most mature industries in Brazil. But for the rest of industries, using retail as an example, it’s not:

There are stores who are already evolving their ecommerces for security, performance, scalability, stability and the most important: user experience. But there’s also those who treat their online stores as something needed. Is common to copy layout practices from concurrents. Also, security is expensive. Then some companies have a budget to lose due to hacker invasions, instead of investing to security.

The good news is that the market is evolving faster and faster. Soon no industry will be behind.

Digital transformation, IT is business, Office

A vision about IT evolution

January 7, 2018

When the IT and the modernization it brought first started arriving the industries, there on the 1960’s, it was faced as something not wanted, but needed. The IT had its own language and terms, like the other areas, but for some reason it was not serious as the other areas. The IT people used to be treated as the people who could make magic and make people’s work easier. But what we have seen lately is a huge evolution and change on how IT is faced.

 

First scenarios

I heard during my graduation, from a professor, the conclusion that IT is a support area only. And I agree it used to be treated like that. The IT team were the strange guys, their workplaces were the worst, even not having windows to look outside. And it can be the reality until today for some companies. But it’s changing fast.

Old IT department

As a software service company employee, for years I used to be called to talk to our clients about new projects. The final conclusion is that all of those meetings were the same. I was called to create something new that could make activities easier and faster to be done by someone. And after that a waterfall move used to happen: less time was needed, then less people were needed, then less money were spent to keep that process/department.

As an instance, for those projects the procedure inside the demanding companies were the same:

  • Identify the need: we are slow at creating contracts for our clients;
  • Start the strategy move to change: let’s automatize the contract parts, that can be automatized, and add collaboration so our writers can work faster and together;
  • Call the IT department: hey, we want a tool!
  • Reach the ROI: we will save US$ 50.000,00 a year having less people writing, and reaching less our law guys;
  • Then the IT department calls the consultancy company (me): please build this tool.

After the tool was delivered, the contracts area got faster, the company saved money, and the whole IT talking was finished. Talking business, our clients were demanding about improving and automatizing parts of processes. The IT used to be faced as the support area which could solve problems or improve processes of the products areas.

 

Then the digital transformation

 

The customer experience focus, is pushing one of the most mature steps of digital transformation we have this far. The IT becoming the companies’ core. This move has been happening based on a world connected to the internet. Since everybody has a smartphone, and everybody want to solve things faster, leaving behind unnecessary stuff (like lines, support waiting times, etc), the IT is the way to reach those people. The customer needs are changing faster and the time spent on that formal process can’t be handled anymore. It must be possible to make changes and adapt at the same speed of the market. Because of that, the IT must be available 100% of the time.

Using the same instance, but now changing to mature business, the new procedure to improve something would be:

  • Identify the need: we must create a new feature to our final customer;
  • Start the strategy move to change: let’s include the feature inside the app;
  • Reach the ROI: we will make US$ 50.000,00 a year with that new feature;
  • Then let’s do it; No more boring meetings are needed;

With this core move, the IT is getting more importance on the business scenario. It’s not the ugly son anymore. The financial industry in Brazil is one of the most mature industries in world, and is pushing the others. I have already seen arguments between the final product areas, which used to rule the scenario, and the IT. It doesn’t make sense anymore to keep these two areas separated.

 

The next industries

This move seen in the financial area, which goes through every single company, isn’t common to all industries yet. For the next industries affected, its easy to understand the more B2C the company is, the faster will be its rush to adapt. Retail is a good candidate to be the next one. But what about the rest? When the factories will get to it?

Avoiding problems, Digital transformation, IT is business, Practical examples

What is telemetry, why it’s important and how to start!

December 17, 2017

The application stability has been a more frequent concern for companies specially when we talk about high value applications. Every time a core application stops working, many money is lost or many money stop being made. Because of that, a lot have been said about telemetry for applications more and more often. But what is telemetry for software actually and how to get benefits from this practice?

 

What is telemetry?

Telemetry is the act of measuring something remotely, by distance, and automatically.

Talking about software architecture, the telemetry is already very easy to find. Some simple examples are the Chrome platform, the Windows, OSX, Android, iOS and Sony’s Playstation OS operational systems, and also your mobile and desktop apps, such as Spotify and Microsoft Office. What these softwares do is to operate and gather all data that matters to its work. Then they send this data, naturally just if you allow, to their manufacturers (Google, Microsoft, Apple, etc). The next step, when they are grouped, is to analyze the data and change things if they have to. The core intention is to improve the systems so they can operate in many different environments having its proper behavior.

So the main thing about telemetry is to operate, gather data, analyze, and then improve the system code to reach a better behavior.

 

Why telemetry?

The telemetry can bring a lot of value to the business. Lets explore an example. Imagine you have an app running, which has a non-interactive FAQ screen to your users. Once your users get there, they will stop using your call center service because they have already found what they were looking for. This means money saving to your company. Now imagine that one of the answers (a quick how-to video) of this screen, for some reason, stops being shown (stops working properly). If you don’t have something checking for this screen’s healthy, it will be hard for you to notice, because we don’t have people browsing on systems 24/7. And then you will depend on some user good will to TELL you that the screen stopped working. It will happen sometime, but before that, your call center will start being called more and more. That’s waste of money because of software bad behavior.

The telemetry can be used to check many levels of service operation:

  • Very deep information, like machine’s CPU and memory. Are they green?
  • Do the number of machines up match your historic knowledge about how many were supposed to be up to support 1, 2 or 3 thousand of users at the same time?
  • Are the core webs-services your customers access every time up?
  • Is the final screen the user uses to login up?

Then when a telemetry practice is watching the important things on your application, you will be able to take actions and prevent problems from happening.

 

How to start?

A good telemetry implementation will depend on the size of information you will want to store and to analyze. The more information you have, the more infrastructure and knowledge you will have to have to process it all. It can even mean using bidata tools. But let’s talk about a new simple example, such as a system that receives vehicles security data from a company that sells insurance letters. A good way to start can be as following:

  • Identify why to measure: let’s measure because it is a core service that tells our customers how their loads are being transported over the roads;
  • Be sure of the goals related: the main goal is to keep the entire system running without down time, because every time this application stops, the contract allows the customers not to pay for that down time;
  • Identify what to measure: let’s check if all of our many data inputs are up. Let’s also check if the inputs are sending the same amount of data they are used to;
  • Set a strategy for measuring: let’s reach every end-point of the many data inputs. If they are available means its healthy. If they are not, its red. Then let’s read the amount of data received in the last minute. If it’s around the known number, its healthy;
  • Set an analysis strategy: can we automatize anything? If one of the endpoints is down, is it useful to restart the operational system, container or application server of the load balancer or the micro services? Or will it have to be shown in a dashboard to a human to analyze?
  • Implement ways to gather the data: let’s create the code to gather data and take actions, or to show it;
  • Show it! Now it’s time to show it. Is it useful to create a chart? Let’s show it using colors, so people can easily identify when there is a problem. If we need results fast, a good very first small step MVP could be opening a ticket on the infrastructure team;
  • Analyze: this is the most important time. It’s time to be critical and identify the root reason of the problem. Do not focus on the problem, but why it is happening. Why it is happening? Do we have problems with the parts that send data to us? Is the problem on our side? If yes, is our application running the way it should? Do we have to change something in our development process?
  • Take actions: through code or not, solve the things you found in all the steps above;

 

The telemetry can bring a lot of value to the business. It will give you intelligence to act before things happen. If your business is critical, it can mean a lot of money easily. It’s a common practice to many things in our administrative areas, like the PDCA mindset thought, why not do that for our software?

Digital transformation

3 steps of Digital Transformation evolution

October 22, 2017

This article was written by me after some insights dropping from discussions with Lucas Lima, a business person who listens and speaks about Digital Transformation almost daily.

 

When we discuss business and companies that are trying to innovate, or are worried about its concurrent’s innovative postures and trying to get head-to-head with them, digital transformation is one of the current most confuse concepts. Every IT consultancy company has it’s “solution” for digital transformation and says that helps companies to innovate and evolve. But is Digital Transformation a single concept to be explored and pursued in just one way?

1st wave: Digital Transformation before 2007:

I’m using the iPhone announcement as a milestone here. Before it and the apps it brought, it used to be very hard and expensive to innovate. The Business Transformation (not Digital yet) used to happen in a much more slow way. It would require a lot of time to be spent on R&D because of different maturity of business areas. The technology available was very limited and it was hard to think about new products without doing a long research. So before a new product announcement, a lot of time had to be spent on business validation, product development, market tests, and the announcement for final users.

After the iPhone release, a wave of initiatives started creating apps to solve small problems of people’s life. They managed to make money, and then the big companies started to notice that. After that process, they realized that this evolution could be done for everything, even when outside a smartphone.

Since then everybody could have the ideas, and the first to launch a solution would be the more successful. We started to head to Digital Transformation as we know it right now.

 

2nd wave: Digital Transformation becomes related to maturity:

Leaving behind those companies who aren’t looking for the evolution of their business and market, I’ve already seen many customers talking about Digital Transformation in some different ways. Each company has an idea about Digital Transformation, even if they don’t call it this way. I’ve listed below few scenarios from less maturity to more maturity talking about the experiences I’ve been having:

  • Digital transforming the communication:

    Picking up a new email. It’s still not rare to find companies who don’t have an stable and reliable email host; Once they still have problems with sending data, too large attachments, spam, live communication tools, shareable agenda, and etc, the problem to be solved here, transforming the organization, is the communication. So, for this example of companies, a new tool to solve all of this problems is Digital Transformation process.

  • Digital transforming the software development methodology:

    Applying agile. Now talking specifically about the IT area of the companies, one of the most known problems of traditional methodologies for software development is the lack of feedback cycles. Once you learn, through feedback, just one time at each project, at the business validation process, it’s pretty common that you have already spent a lot of money to reach that point and discover that the software is not what you needed, or it’s already out of date. When you transform your IT area to apply an agile methodology set of practices focusing on acknowledging feedback at the right time, it’s pretty easier and cheaper to develop software and then to support your business areas demands. So, what this example company calls a Digital Transformation is a very specific internal process.

  • Digital transforming the business through DevOps or DevSecOps:

    Gathering everything that matters to work faster. When the company works being guided by departments which do only what they are supposed to and not getting the whole picture of their importance for the business, the natural way is to create a lot of bureaucracy and start getting slower and slower. Because of the new startups market movements, the old big companies started having to evolve internally. They have the money, the market and the intention not to turn obsolete. All they have to do is to change their behavior and mindset. When the maturity of people inside the companies allow them to put everybody who’s important to the business to work together, is when they will get real speed to innovate. If they manage to get business, marketing and sales guys (who really think about getting money, and strategies, and etc), to work together with UX guys (who will care about what the customers think and feels about the products) and the IT guys (who will actually develop and deliver the products), they will transform how the company create products, innovate and affect the market.

  • Digital transforming the business through 100% Lean process:

    The nirvana. You are a startup and you want to create something disruptive to the market. It’s easier to be 100% Lean because you will probably have few people with you. But take care because as soon as your product start growing, it will be easy to loose your lean mindset.

 

3rd wave: future of Digital Transformation:

Very soon, not more than 5 years ahead, we will be reaching a stage where all of the concurrents, considering startups and big companies, will have knowledge and internal process as mature as they need to, so they will be able to be very fast to have ideas, prototype it, test and launch it. The new rush will be for the technologies that will allow the creation of those new products.

I’ve seen many different technologies with a potential of changing the world:

  1. The Blockchain can turn banks useless in few decades, since we wouldn’t have to care about who “holds” our money, killing one of the oldest markets we have.
  2. IoT and Cognitive services have the potential to put an end to an entire class of labor, turning people’s life easier and pushing those workers to new professions.
  3. AR/VR applications can change how we interact with everything. Not having to be physically somewhere, but having an immersive experience, is almost like teleporting.

Along with that, we still have many markets not explored yet. Some examples:

  1. Textiles. It’s been the same thing for the last 500 years to make textiles. Google has Project Jacquard, which is a nice step. But is it everything we can do?
  2. Public security. We do have the cameras, the software to find people and the problem of security. So why not monitor everything and everybody?
  3. Taxes. Why do we still have to do our yearly Income Taxes report? The government already has all the data.

 

The future of Digital Transformation will continue to happen faster and faster. The companies who manage to find the opportunities emerging from the new technologies will be the successful ones.

Customers, Digital transformation, Entrepeneurship

The innovation wave we’re not surfing well

October 1, 2017

During the last few years, we have seen many new and disruptive companies appearing around us. They are called “Startups”. They are understanding the trends and the new possibilities the market and the technology are providing, finding spaces to establish themselves, and staying untouched for some time, what allows them to keep innovation, growing and pushing the old companies.

The wave

This very fast innovation and market disruption is a no turning back movement, and the companies and people will innovate even faster on the next years. But the successful companies we have seen, such as Contabilizei, Guia Bolso, Love Mondays, Movile and Nubank, who managed to grow, are just a very small group of a horde of companies. By the start of 2016, Brazil had 4151 startups (which was a big number for us). But on that same time, just the Silicon Valley area, on the US, had more than 23000 startups.

So why Brasil, one of the most creative countries in world, counting with the sixth biggest population number, has a very short number of startups being created and succeeding? Do we have the ideias? Are they being well explored?

 

Where we are stopping

It’s well known that Brasil has the bureaucracy within its culture, and this data from the World Bank Group shows that, compared to the US, its 15 times harder to do business here:

Source: World Bank Group

 

But is it just the bureaucracy that forbids us to create new business, markets and innovate in large scale? Maybe our way to approach problems are not well mature as other countries’.

As an known example, we have the factories of hair nets: by the 1910’s decades, the business were to produce more and more hair nets so the women on that time could hold their hair to keep the hairstyle. The factory owner kept that in mind and couldn’t see the real question. The demand were not about hair nets. The demand were about holding hair. Then the hair fixative came and the industry of hair nets went bankrupt. But why the hair net’s industry owner didn’t see the market change? His mindset were adjusted to the product, not to the demand. Not to the customer.

Mindset has to be “customer first”

Now approaching the same issue with a service-oriented view. A regular hair style shop hierarchy would be something like:

On this diagram, who is the most important actor TO THE CUSTOMER? Many would say the manager or even the owner is the most important. They have autonomy to solve any issues, and they know best how to treat customers and how to make them happy, because they already have much experience. But the reality is that, since our customer is getting more and more demanding, the owner AND the manager are just useless. The attendants are the most important part here! They have a key behavior to the whole system. Let’s think about the customer getting to the hair style shop and having problem with the attendant. The manager will be called and the problem will be solved. Alright! It’s solved but it was a reactive movement. Depending on the stress level caused to the customer, it is very probable he won’t get back to the hair style shop. He will manage to find a better service.

 

Business strategy

So a better approach can be to think about the attendants as the most important part of the system.

But the owner and managers still are bosses… what changes here? The mindset has to change. The attendant who is interacting with all the customers must understand the company goals and all of it’s responsibilities. He has to have the power and autonomy to solve any issues, so no customer would want to talk to someone else. All of its demands would be solved as fast as possible. The manager role is simplified as someone who knows the whole company and can think about the strategies and make changes and fixes when they are needed.

As business strategy we must think like what is the real customer demand. Does he really needs a hair cut? Or does he wants to look handsome? Maybe when the biggest part of Brazil entrepeneurs change their mindset to this, we will start having a largest number of companies being created.

Customers, Digital transformation

The eXperience as business strategy

September 9, 2017

This article is a free translation, with some updates, from this original one that was wrote by me and Felipe Santos.

 

UX aggregated value has changed software development market

The companies that produce software and don’t care about the end-user experience have their days counting down. The user is becoming more demanding (picky, why not?) and, because of that, tolerates just a few number of failures of systems and processes. Once they can’t complete a purchase, signature of something, or any activity that depends on a system, there’s a high chance that they will give up and start looking for a competitor. According to a Harvard research, brazilian retail executives are the biggest foreign crew attending to international events about UX (User eXperience), looking for innovation on their market. This information defines very well the business strategy that have been pushing our country, and tells us that it is very important to guarantee the best customer experience.

Once we think about user experience, we have already seen extremely complex applications, that used to require training for people to learn how to operate them. Those applications are very rare right now. Processes that used to be very detailed and manual, forcing the user to repeat information in different areas of the system, are being automated. The UX increasing value transformed our market, from one complex view to one simple look. Everybody have been discussing about a positive experience as part of business strategy.

 

Relationship is the answer

The transformation on software development, to offer a better user experience, is a result of an evolution on the user relationship to the technology and on the level of consumer demands. The idea is common once we analyse the evolution of some startups, that managed to transform a whole area of their market. We can find good examples of that on banking and transport areas, such as Nubank, Neon, 99 and Uber. What do they have in common? They started offering exactly the SAME products and services that were already available on the market, but with one big difference: the experience. The focus on the best ever possible relationship to their consumers.

Keeping a good user experience, from the very first moment, when they get to know your company, until they finish a transaction, is very complex. According to a research, conducted by Compuware, only 16% of users will try to use an app for the third time, if they fail for the first two times.

The user sees all the steps where he interacts with a product or service as just one entity, and the company absorbs this experience results directly. Some good examples of that bad interactions are the call centers, that redirects the user many times before solving the problem and create differences between the company’s areas, like the “new business area”, “payments”, “operations” or “complaints”. But for the user, there’s just one single experience being lived across the whole interaction. And as a result, it may happen that they think twice about a new interaction with that company, or even get a bad feeling about the brand.

Because of that, all the steps must be integrated and the process must be simple. At ilegra, our projects are planned and conducted in an iterative and incremental way, including research and constant validations until the adoption of the best market practices and technologies. A good example of those technologies are, the just released, Angular 4, and React web or Native, with which we have been working on MVPs with the internal teams. It all focuses on the best possible experience with our users, during all the process parts.

 

On and Off: one single experience

The user complete experience goes all the way through online and offline interactions. It’s important to know that there’s no difference between the different communication channels, because the message is received in one single way by the customer: either the legacy is positive or negative. On “online”, the applications or web systems, even if they are brand new, they may present many bad steps, such as repetitions, slow processes, a lot of information requests, etc. On “offline” interactions, physical or by phone, it also may cause frustration, such as not solving a problem or even just spending too much time waiting for attendance.

These examples, on both online and offlice channels, severely impact the user perception and its loyalty to the brand or company. Its not just ilegra that have seen this trend. Its remarkable, on corporations market, to see the user experience excellence as a trend and target for software development on 2017. Because of that, it’s the right time for traditional companies to get on board of this new moment, evolve and innovate. This is the point that will determine the companies that will grow on their markets.

We can guarantee, by the critical view and experience that we apply to all of our projects, that to provide a good user experience is needed dedication and processes. Terms like co-creation, systemic thinking, discovering, immersion, among many others, must get out of paper and be followed by the teams.