What is a JSON feed? Learn more

JSON Feed Viewer

Browse through the showcased feeds, or enter a feed URL below.

Now supporting RSS and Atom feeds thanks to Andrew Chilton's feed2json.org service

CURRENT FEED

Stories by Ravindu Senaratne on Medium

Stories by Ravindu Senaratne on Medium

A feed by yourfriends@medium.com

XML


Object Detection with Flutter and TensorFlow Lite

Permalink - Posted on 2020-09-22 13:17

How to develop an object detector mobile application with Flutter, using TensorFlow Lite and SSD MobileNet

Photo by Debby Hudson on Unsplash

In the previous article of this series on developing Flutter applications with TensorFlow Lite, we looked at how we can develop digit recognizer with Flutter and TensorFlow Lite and image classification with Flutter and TensorFlow Lite.

In the third article of this series, we’ll keep working with TensorFlow Lite, this time focusing on implementing object detection. The application we are going to build will be able to recognize objects presented in an image.

Application and Use cases

TensorFlow Lite gives us pre-trained and optimized models to identify hundreds of classes of objects including people, activities, animals, plants, and places. Using the SSD MobileNet model we can develop an object detection application.

Required Packages

  • TensorFlow Lite

tflite | Flutter Package

  • Image Picker

image_picker | Flutter Package

  • SSD MoblieNet (tflite) Model

The SSD MobileNet model is a single shot multibox detection (SSD) network intended to perform object detection.

ravindu9701/tflite-models

Download the tflite folder from the above link, and put it inside the assets folder of the Flutter project.

Kickstart your mobile machine learning projects — subscribe to the Fritz AI Newsletter for looks at common mobile ML problems, helpful resources, and much more.

Flutter Application

Now that we have put the model inside the project folder, we can develop our Flutter application to detect objects. Let’s get started.

We’ll need to initiate five variables to carry out the application properly. To receive the output, we’ll need a List variable, to get the uploaded image we’ll need a File variable, to get the image’s height and width we’ll need two double variables, and to manage errors we’ll need a Boolean variable:

File _image;

double _imgWidth;
double _imgHeight;
bool _busy = false;

List _recognitions;

Since the model is working offline, we need to load the model when the application is launched.

loadModel() async {
Tflite.close();
try {
await Tflite.loadModel(
model: "assets/tflite/ssd_mobilenet.tflite",
labels: "assets/tflite/ssd_mobilenet.txt",
);
} on PlatformException {
print("Failed to load the model");
}
}

Using the above function we can load the model. First, we need to close any other running models using Tflite.close(). Then we can load our model inside a try-catch.

selectFromImagePicker() async {
var image = await ImagePicker.pickImage(source: ImageSource.gallery);
if (image == null) return;
setState(() {
_busy = true;
});
predictImage(image);
}

In the above function, we have used ImagePicker to pick an image from the phone’s gallery. After receiving an image we need to pass that image through the model. In order to do that, I have created another function called predictImage(image) and passed the selected image into it.

predictImage(File image) async {
if (image == null) return;
await ssdMobileNet(image);

FileImage(image)
.resolve(ImageConfiguration())
.addListener((ImageStreamListener((ImageInfo info, bool _) {
setState(() {
_imgWidth = info.image.width.toDouble();
_imgHeight = info.image.height.toDouble();
});
})));

setState(() {
_image = image;
_busy = false;
});
}

Using the above function’s FileImage we can get the image’s dimensions using listeners. The image will be passed on to the ssdMobileNet() function to run the image through the model.

ssdMobileNet(File image) async {
var recognitions = await Tflite.detectObjectOnImage(
path: image.path, numResultsPerClass: 1);

setState(() {
_recognitions = recognitions;
});
}

When we use the SSD MobileNet model, we don’t need to give much information because it already has them as its default settings. We need to give the image path and the number of results per class.

Now we need to create a function to get detection boxes (to draw the boxes on the top of the image) which will get the size of the screen as an input.

List<Widget> renderBoxes(Size screen) {
if (_recognitions == null) return [];
if (_imgWidth == null || _imgHeight == null) return [];

double factorX = screen.width;
double factorY = _imgHeight / _imgHeight * screen.width;

return _recognitions.map((re) {
return Positioned(
left: re["rect"]["x"] * factorX,
top: re["rect"]["y"] * factorY,
width: re["rect"]["w"] * factorX,
height: re["rect"]["h"] * factorY,
child: Container(
decoration: BoxDecoration(
border: Border.all(
color: Colors.red,
width: 3,
)),
child: Text(
"${re["detectedClass"]} ${(re["confidenceInClass"] * 100).toStringAsFixed(0)}%",
style: TextStyle(
background: Paint()..color = blue,
color: Colors.white,
fontSize: 15,
),
),
),
);
}).toList();
}

First, we need to check that our _recognitions is not null. Secondly, we need to check the image width and height to see whether they are not null. Then we need to find the factorX and factorY using the above formula. Finally, we need to map the recognitions using interpolation. Now inside the child, we can define the name of the object and the confidence percentage.

Now that we’ve identified the required functions, we can develop the UI so that our app can actually surface these results to the user:

@override
Widget build(BuildContext context) {
Size size = MediaQuery.of(context).size;

List<Widget> stackChildren = [];

stackChildren.add(Positioned(
top: 0.0,
left: 0.0,
width: size.width,
child: _image == null ? Text("No Image Selected") : Image.file(_image),
));

stackChildren.addAll(renderBoxes(size));

if (_busy) {
stackChildren.add(Center(
child: CircularProgressIndicator(),
));
}

return Scaffold(
appBar: AppBar(
title: Text("Object detetction"),
backgroundColor: Colors.red,
),
floatingActionButton: FloatingActionButton(
child: Icon(Icons.image),
backgroundColor: Colors.red,
tooltip: "Pick an Image",
onPressed: selectFromImagePicker,
),
body: Stack(
children: stackChildren,
),
);
}

Results

Now that we’ve implemented the Flutter application code, let’s look at the output of the application when it’s up and running:

Example

Full code:

Source Code:

ravindu9701/Obj-detect-flutter-tflite

Conclusion

You can use other models that are compatible with TensorFlow Lite such as YOLOv2, v3, v4, Pose_net, and Deeplabv2 to develop your own object detection mobile application. These models have their own perks and accuracy levels which would be beneficial to the goal you're trying to achieve.

That’s all for this article, but in the next article, I’ll try to integrate real-time object detection using the mobile camera.

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.


Object Detection with Flutter and TensorFlow Lite was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.


Image Classification on Mobile with Flutter, TensorFlow Lite, and Teachable Machine

Permalink - Posted on 2020-09-14 13:26

Develop an image classifier mobile application with Flutter, using TensorFlow Lite and Google’s Teachable Machine

In the previous article of this series on developing Flutter applications with TensorFlow Lite, we looked at how we can develop a Digit Recognizer using TensorFlow Lite.

In the second article of the series, we’ll keep working with TensorFlow Lite, this time focusing on implementing image classification to classify images between two classes. The application we are going to build will be able to classify whether an input image contains a horse or a human.

Application and Use Cases

TensorFlow Lite gives us pre-trained and optimized models to identify hundreds of classes of objects, including people, activities, animals, plants, and places. Using Teachable Machine from Google, we can develop our own custom model using some of our own images. Teachable Machine 2.0 allows you to train machine learning models in the browser, without any ML code. You can train models on images, sounds, and poses, and then you can save the models you trained and use them on your own projects.

Required Packages

  • TensorFlow Lite

tflite | Flutter Package

  • Image Picker

image_picker | Flutter Package

  • Horse or Human Dataset

The set contains 500 rendered images of various species of horses, in various poses, and in various locations. It also contains 527 rendered images of humans in various poses and locations:

Horses Or Humans Dataset

This dataset will be used to create the model using the Teachable Machine. First, download the dataset from the above link.

Training a model with Teachable Machine

Before we train our data using the Teachable Machine, we need to gather the required data for the machine to learn. The data collection process hinges on the problem you're trying to solve. For image classification problems, after this data is collected, it needs to be arranged into relevant classes (i.e. labeled). This is where the Teachable Machine comes in.

After we’ve downloaded the required data through the Kaggle, we need to train a TensorFlow Lite model on it.

For this application, we need to divide our dataset into 2 classes. (Horses and Humans). The downloaded dataset from the Kaggle contains two different folders, depending on the class. We need to provide these images folders to the Teachable Machine and label them as follows:

The second step is to train the model, which you can do by clicking the Train Model button. It will take a few minutes to train the model. You can try Advanced settings and give a custom number of epochs (basically, the time to train the model will depend on the number of epochs you define).

After the training is complete, you will be able to click the Export Model button.

Download the model for TensorFlow Lite with the model conversion point Floating Point.

A quick look at all of your options here:

  • Floating Point — Qualified for CPU and GPU (2x smaller, GPU acceleration)
  • Quantized — Qualified for CPU (4x smaller, 2x-3x speedup)
  • EdgeTPU — Qualified for EdgeTPU and Microcontrollers (4x smaller, 3x speedup)

Flutter Application

Now that we have the trained model, we can develop our Flutter application to classify an image as including either a horse or a human. Let’s get started.

First, create an assets folder inside the project folder and copy the downloaded model into that folder.

Since this is a beginner-level application, we can write the code inside the main.dart file without creating another page.

We’ll need to initiate three variables to carry out the application properly. To receive the output, we’ll need a List variable; to get the uploaded image we’ll need a File variable, and to manage errors we’ll need a Boolean variable:

List _outputs;
File _image;
bool _loading = false;

Since the model will also be working offline (one of the benefits of on-device machine learning), we need to load the model when the app is launched:

loadModel() async {
await Tflite.loadModel(
model: "assets/model_unquant.tflite",
labels: "assets/labels.txt",
);
}

Using the function above, we can load the model. To run this function when the application is getting started, we can call this function inside the initState() method:

pickImage() async {
var image = await ImagePicker.pickImage(source: ImageSource.gallery);
if (image == null) return null;
setState(() {
_loading = true;
_image = image;
});
classifyImage(image);
}

With the above function, we can use the ImagePicker to pick an image from the gallery. We can then pass this image on to the classifyImage() function to run it through the classification model:

classifyImage(File image) async {
var output = await Tflite.runModelOnImage(
path: image.path,
numResults: 2,
threshold: 0.5,
imageMean: 127.5,
imageStd: 127.5,
);
setState(() {
_loading = false;
_outputs = output;
});
}

Now that we’ve identified the required functions, we can develop the UI so that our app can actually surface these results to the user:

@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
backgroundColor: Colors.red,
title: Text('Horse/Human Detector'),
),
body: _loading
? Container(
alignment: Alignment.center,
child: CircularProgressIndicator(),
)
: Container(
width: MediaQuery.of(context).size.width,
child: Column(
crossAxisAlignment: CrossAxisAlignment.center,
mainAxisAlignment: MainAxisAlignment.center,
children: [
_image == null ? Container() : Image.file(_image),
SizedBox(
height: 20,
),
_outputs != null
? Text(
"${_outputs[0]["label"]}",
style: TextStyle(
color: Colors.black,
fontSize: 20.0,
background: Paint()..color = Colors.white,
),
)
: Container()
],
),
),
floatingActionButton: FloatingActionButton(
onPressed: pickImage,
backgroundColor: Colors.red,
child: Icon(Icons.image),
),
);
}

Results

Now that we’ve implemented the Flutter application code, let’s look at the output of the application when it’s up and running:

Example

Full code:

Source code:

ravindu9701/Image-Classification-Flutter-TensorFlow-Lite

Conclusion

Once you get the hang of it, you can see how easy is it to use TensorFlow Lite with Flutter to develop proof-of-concept machine learning-based mobile applications. Teachable Machine from Google Creative Labs helps us create custom models pretty easily and without much ML expertise. Moreover, you can visit Kaggle and download various datasets to develop different classification models.

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.


Image Classification on Mobile with Flutter, TensorFlow Lite, and Teachable Machine was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.


Digit Recognizer with Flutter and TensorFlow Lite

Permalink - Posted on 2020-08-20 13:44

How to develop a digit recognizer mobile application with Flutter, using TensorFlow Lite and the MNIST dataset

Photo by Andrew Buchanan on Unsplash

Machine learning and AI are taking mobile application development to a new level. Apps that utilizing machine learning can recognize speech, images, and gestures. This gives us new and compelling ways to engage and interact with people in the world around us. But how do we integrate machine learning into our mobile apps?

Developing mobile applications that incorporate machine learning has long been a difficult task. But with the help of platforms and dev tools such as Fritz AI, Firebase’s ML, and TensorFlow Lite, it’s getting easier to do so. These tools provide us with pre-trained machine learning models as well as tools to train and import our own custom models. But how do we actually develop a compelling experience on top of those machine learning models? That’s where Flutter comes in.

The Flutter SDK is a portable UI toolkit built by Google and its open-source community to develop applications for Android, IOS, Web, and Desktop. At its core, Flutter combines a high-performance graphic engine with the Dart programming language. Dart provides both robust type safety and stateful hot reload, which helps developers build reliable apps quickly. Using Flutter, we can build mobile apps with machine learning capabilities like image classification and object detection, for both Android and iOS platforms.

Editor’s note: For a deeper dive into what’s possible with machine learning on mobile, check out our free ebook exploring 14 real-world use cases.

In this article, we’ll combine the power of Flutter and on-device ML to develop a Flutter application that can recognize handwritten digits, using TensorFlow Lite and the famous MNIST dataset.

Required Packages

  • TensorFlow Lite

tflite | Flutter Package

In the android/app/build.gradle file, add the following setting in the android block and change the miniSdkVersion to 19. The installation walkthrough is given in the above Flutter package.

aaptOptions {
noCompress 'tflite'
noCompress 'lite'
}
  • MNIST (Modified National Institute of Standards and Technology database) dataset: One of the largest databases of handwritten digits, commonly used to train image processing systems.

Introduction to CNN Keras - 0.997 (top 6%)

Download the mnist.tflite file from the above link, and put it inside the assets folder of the Flutter project.

Create a mnist.txt file inside the assets folder and add the appropriate Model labels. (0 to 9 numbers)

Now that we’ve taken a quick look at how to set up your Flutter app to run TensorFlow Lite, let’s look at how to get it up and running.

Before we get started, we need to create 4 packages, models, screens, services, and utils.

folder structure

I’ll show the required code snippets for the above files.

In the above code snippet, We use the MNIST dataset of handwritten digits and create an application with a canvas where the user draws the digits.Widget_drawCanvasWidget() is used to determine the size of the drawing canvas. After defining the size of the draft we need to pass the data to using theCustomPaint(painter: DrawingPainter). Through drawing_painter.dart we can get the canvas details which contains properties for the canvas (brush size, color, background-color, etc.).

After the drawing (i.e. after stopping the drawing) the onPanEnd attribute calls the recognize() class and passes the image data. This will be used inside the service -> recognizer.dart.

The Recognizer class will load the mnist.tflite model and the mnist.txt label file. This will then be run through the image data we pass when we draw on the canvas. It’ll check for the similarity of the digit the dataset contains and pass the input through the prediction.dart class to convert the index, confidence, and label to JSON format. Then, we pass this to the prediction_widget.dart to display the most accurate digit.

prediction_widget.dart contains the styles and numbers required to show the predicted digit for the one we drafted as input.

Now that we’ve taken a look inside the Flutter application code, let’s look at the output of the application when it’s up and running.

Example

Source Code:

ravindu9701/digit_recognizer

Conclusion

Once you get the hang of it, you can see how easy is it to use TensorFlow Lite with Flutter to develop proof-of-concept machine learning mobile applications. To improve your knowledge, you can visit the Kaggle site and download various datasets to develop different classification models.

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.


Digit Recognizer with Flutter and TensorFlow Lite was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.


How to Augmentate Data Using Keras

Permalink - Posted on 2020-07-21 16:17

Data augmentation using Python and Keras


Make Your Own Deepfake Video in a Few Easy Steps

Permalink - Posted on 2020-07-13 13:41

A simple guide for creating your own Deepfake video

Photo by Christian Gertenbach on Unsplash

If you’re reading this article, then you’ve likely seen or heard about a particular AI algorithm called deepfakes. Much of this hype has comes as a result of major apps like Snapchat, FaceApp, or Instagram using this algorithm to create entertaining content.

In this article, I’ll show you how to quickly and easily create your own deepfake video. To do this, I’ll be using the implementation for the research effort First Order Motion Model for Image Animation, developed by Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe.

I'll show you how to use their code to create deepfakes using your own pictures and videos.

What is Deepfake?

A deepfake video is a fake photo, video, or story generated by neural networks. Deepfake developers use artificial intelligence and machine learning algorithms to imitate the facial features and body postures of real humans.

These deepfakes diverge from other common forms of fake media by being extremely hard to identify. In addition to their impressive technological underpinnings, deepfakes have the potential to cause enormous damage.

In April 2018, BuzzFeed showcased how far deepfake video technology has come by combining Barack Obama’s face with Jordan Peele’s convincing voice impersonation.

Deepfakes depend on autoencoder neural network type. It contains an encoder, which decreases an image to a lower-dimensional latent space, and a decoder, which reconstructs the image from the latent representation. Deepfakes use this structure by having a universal encoder which encodes a character into the latent space. The latent representation includes key characteristics of their facial features and body posture.

Now that we’ve taken a quick look at deepfakes and some of the controversy surrounding them, let’s look at how to implement our own.

The future of machine learning is on the edge. Subscribe to the Fritz AI Newsletter to discover the possibilities and benefits of embedding ML models inside mobile apps.

For setting up our deepfake, I’ll be using Aliaksandr Siarohin’s Google Colab code.

First, make sure you copy his code to your Google Drive since we don’t want to alter his code:

Google Colaboratory

For a faster compile time go to “Runtime -> Change runtime type -> Select GPU

Before we get started, we need to download a few required files from the Drive link below. You just need to download one .jpeg image, one .mp4 video, a vox-cpk.pth.tar file, and a vox-adv-cpk.pth.tar file. But if you need to check with the images and videos they provided, download everything.

first-order-motion-model - Google Drive

After downloading the files, create a folder named first-order-motion-model in your Google Drive and upload the downloaded files.

To run with your own data…

  • If you want your face to be added to a video, then take a selfie and change its size to 256 X 256 pixels (you can simply use Paint to edit the size). This is the input size that the model accepts.

Upload all your data to the first-order-motion-model folder on your Drive.

To run your face on the videos they’ve provided, change the following code to your file name. Then follow the rest of the steps as presented.

source_image = imageio.imread('/content/gdrive/My Drive/first-order-motion-model/*File Name.png*')
  • If you want someone else’s face to be added to your video file, that file also needs to be on the size of 256 X 256 pixels.

To convert your video into the above size, scroll down to the part of the code labeled “Run on your data”. Remove the boilerplate code provided and add the following code:

!ffmpeg -i /content/gdrive/My\ Drive/first-order-motion-model/*File Name.mp4* -s 256x256 -c:a copy result.mp4

This will create a converted video result.mp4

Copy the result.mp4 file into thefirst-order-motion-model folder and change the following code from “Load driving video and source image”

driving_video = imageio.mimread('/content/first-order-model/result.mp4')

And that’s it. Here is an example I created using Nicolas Cage’s face and the TV show Friends.

Example

I hope this quick start guide to setting up your very own deepfake was useful. Have fun making some funny content (nothing dangerous!). Thank you, and stay safe!

Resources

AliaksandrSiarohin/first-order-model

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.


Make Your Own Deepfake Video in a Few Easy Steps was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.


Using DeOldify to Colorize and Restore Grayscale Images and Videos

Permalink - Posted on 2020-07-06 13:26

Bring back missing colors using DeOldify

Photo by Daniel Curran on Unsplash

Image colorization is an engaging topic in the field of image-to-image translation. Even though color photography was invented in 1907, It didn’t become popular for the average person until the 1960s because of its expensiveness and inaccessibility. All the photography and videography up until then was done on Black & White. Colorizing these images was impossible—until the DeOldify deep learning model came to life.

What is DeOldify?

DeOldify is an open-source deep learning model created by Jason Antic, used to add high-quality colorization to grayscale images with amazing results. Simply put, the goal of this deep learning model is to colorize, restore, and give new life to old images and film footage.

Before we get started on the installation, let’s see an example of how it might look when we use DeOldify to colorize old film footage.

The old version of DeOldify used GANs (Generative Adversarial Networks), a network architecture that includes repetitive interaction between two complementary neural networks: a generator and discriminator. The pre-trained generator model does the task of adding colors to the recognized objects, and the discriminator does the task of “criticizing” the generator’s choice of color.

After the introduction of NoGAN training (covered in more details below) and combining it with GAN training, DeOldify was able to produce more stable and colorful images.

What is NoGAN?

To address stabilization problems in the previous DeOldify model, Jason Antic developed this new type of GAN training, which requires less time on GAN training and more time on pre-training the generator and discriminator separately with uncomplicated, fast, and reliable methods.

Effect of using NoGAN

Now that we’ve taken a quick look at how the inside of DeOldify works, let’s look at how to get it up and running.

Machine learning is rapidly moving closer to where data is collected — edge devices. Subscribe to the Fritz AI Newsletter to learn more about this transition and how it can help scale your business.

Setting Up DeOldify

Method 1: Using Google Colab

The easiest and simplest way of trying out DeOldify is to use the Google Colab versions.

  • Image (Artistic)

Google Colaboratory

  • Video (Artistic)

Google Colaboratory

If you’re having trouble with the artistic image colorizer, try the “stable” version below. It usually won’t present colors that are as attractive as on the Artistic version, but the glitches are noticeably reduced.
  • Image (Stable)

Google Colaboratory

Method 2: Using the Jupyter Notebook

To use a Jupyter Notebook, first, you need to clone the DeOldify repository.

git clone https://github.com/jantic/DeOldify.git DeOldify

Next, create a folder called models inside the DeOldify folder and download the pre-trained weights into that folder.

cd DeOldify
mkdir models
  • For Image Colorizer (Artistic):
!wget https://www.dropbox.com/s/zkehq1uwahhbc2o/ColorizeArtistic_gen.pth?dl=0 -O ./models/ColorizeArtistic_gen.pth
  • For Video Colorizer (Artistic):
!wget https://www.dropbox.com/s/336vn9y4qwyg9yz/ColorizeVideo_gen.pth?dl=0 -O ./models/ColorizeVideo_gen.pth
  • For Image Colorizer (Stable):
!wget https://www.dropbox.com/s/mwjep3vyqk5mkjc/ColorizeStable_gen.pth?dl=0 -O ./models/ColorizeStable_gen.pth
If an error throws (!wget), manually download the pre-trained weights using the above links.

After configuring the pre-trained weights, we need to install the required dependencies:

pip install -r requirements.txt

Once the installation is done, open the project using the Jupyter Notebook. All the necessary steps are commented on the .ipynb file and you can continue from there onwards.

Conclusion

I hope this guide to setting up DeOldify on your system was useful. Below is an example of DeOldify that I particularly like. Have fun colorizing your favorite grayscale content. Thank you and stay safe!

The Kid 1921 (Charlie Chaplin Movie) — Colorized

Resources

jantic/DeOldify

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.


Using DeOldify to Colorize and Restore Grayscale Images and Videos was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.


CLAHE and Thresholding in Python

Permalink - Posted on 2020-07-03 16:36

Contrast Limited Adaptive Histogram Equalization and Thresholding using OpenCV in Python


Social Distance Detector with Python, YOLOv4, Darknet, and OpenCV

Permalink - Posted on 2020-06-29 12:51

Learn how to create a social distance detector using YOLOv4, Darknet, CUDA, and OpenCV

As the spread of COVID 19 continues, communities are told to reduce close contact between individuals. This is called Social Distancing, as it is a necessary and effective way to slow down the spread of the virus. As a data science student, I came with a solution to identify whether people actually follow the social distancing protocol of staying at least 6 feet from each other.

Overview

What YOLOv4?

YOLO stands for You Only Look Once. It’s a fast operating object detection system that can recognize various object types in a single frame more precisely than other detection systems.

According to their documentation, it can predict up to 9000 classes. YOLO is built on a single CNN (convolutional neural network). The CNN separates an image into regions and calculates its bounding box and probabilities for each region. In YOLOv4, it uses CSPDarknet53 (CNN enhancement for increasing the learning capability), which is a lot faster than EfficientDet (YOLOv3 CNN enhancement), MaskRCNN, and RetinaNET.

Because YOLO can be used with a conventional GPU, it provides widespread adoption, faster FPS, and more accuracy.

Prerequisites:

Implementation, in 5 Steps:

  1. Calculate Euclidean distance between two points
  2. Convert center coordinates into rectangle coordinates
  3. Filter the person class from the detections and get a bounding box centroid for each person detected
  4. Check which person bounding boxes are close to each other
  5. Display risk analytics and risk indicators

Let’s Get to Coding… 💻

Step 1. Calculate Euclidean Distance of Two Points

We need to calculate the Euclidean distance in order to identify the distance between two bounding boxes. The function is_close gets two points, p1 and p2, as inputs for calculating the Euclidean distance and returns the calculated distance dst.

Step 2. Converts Center Coordinates into Rectangle Coordinates

The function convertBack gets parameters x, y—the midpoint of the bounding box—and wand h—the width and height of the bounding box—as inputs. Then it will covert the center coordinates to rectangle coordinates and return the converted coordinates, xmin, ymin, xmax, and ymax.

The last 3 steps are written inside the cvBoxes function.
Machine learning is rapidly moving closer to where data is collected — edge devices. Subscribe to the Fritz AI Newsletter to learn more about this transition and how it can help scale your business.

Step 3. Filtering the Person Class from Detections and Getting a Bounding Box Centroid for Each Person Detection

First, we check whether there is any detection in the image using a len(detections)>0 condition. If the condition succeed, we create a dictionary called centroid_dict and a variable called objectId set to zero. Then it will run through another condition that filters out all the other detection types, except for person. We’ll store the center points of all the person detections and append it to the bounding box for persons detected.

Step 4. Check which person bounding boxes are close to each other

First, we create a list that contains all object IDs of the under-threshold distance conditions. Through iteration, we can get all the combinations of close detection to calculate the Euclidean distance using the function from Step 1.

Then we set the social distance threshold (75.0) (equivalent to 6 feet) and check whether it satisfies the condition distance < 75.0. Finally, we set colors for the bounding boxes. Red for an at-risk person and Green for a protected person.

Step 5. Display risk analytics and risk indicators

In this final module, we indicate the number of people at risk in a given frame. Starting from the text we want to display with a counted number of people at risk. For better graphical representation, Using for check in range, We tell the application to draw the lines between the nearby bounding boxes and iterate through the red_line_list.

Let’s see the demo:

Running on Nvidia GTX 1050 TI

ravindu9701/Social-Distance-Detector

Conclusion

I hope that this has given you enough knowledge into developing your own social distance detector by using any object detector. For more information on YOLO, you can check out the official docs.

That’s it for the article. Thank you for spending your valuable time reading the article. Check my other articles as well. Stay safe!

Resources

Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.

Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.

If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.


Social Distance Detector with Python, YOLOv4, Darknet, and OpenCV was originally published in Heartbeat on Medium, where people are continuing the conversation by highlighting and responding to this story.


Object Detection on Python Using Template Matching

Permalink - Posted on 2020-06-28 02:09

Learn how to do object detection on python without machine learning or any frameworks


10 Awesome Open-Source Deep Learning Projects

Permalink - Posted on 2020-06-26 15:11

List of awesome open-source deep learning projects you must try in 2020