Active10 months ago
I have a web site that allows users to upload images of cars and I would like to put a privacy filter in place to detect registration plates on the vehicle and blur them.
To compile this example you will need Microsoft Visual Studio installed on your computer. Corporate use of license plate recognition. License plate recognition is an extremely popular function nowadays. You can benefit from it if you wish to analyze and detect license plates in a.
The blurring is not a problem but is there a library or component (open source preferred) that will help with finding a licence within a photo?
Caveats;
- I know nothing is perfect and image recognition of this type will provide false positive and negatives.
- I appreciate that we could ask the user to select the area to blur and we will do this as well, but the question is specifically about finding that data programmatically; so answers such as 'get a person to check every image' is not helpful.
- This software method is called 'Automatic Number Plate Recognition' in the UK but I cannot see any implementations of it as libraries.
- Any language is great although .Net is preferred.
50522 gold badges1212 silver badges2626 bronze badges
Ryan O'NeillRyan O'Neill3,42244 gold badges4242 silver badges5959 bronze badges
closed as too broad by Samuel Liew♦Nov 27 '18 at 2:17
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
12 Answers
I coded a C# version based on JAVA ANPR, but I changed the awt library functions with OpenCV.You can check it at http://anprmx.codeplex.com
Jivan MirandaJivan Miranda
EDIT: I wrote a Python script for this.
As your objective is blurring (for privacy protection), you basically need a high recall detector as a first step. Here's how to go about doing this. The included code hints use OpenCV with Python.
- Convert to Grayscale.
- Apply Gaussian Blur.
Let the input image be the following.
- Apply Sobel Filter to detect vertical edges.
- Threshold the resultant image using strict threshold or OTSU's binarization.
- Apply a Morphological Closing operation using suitable structuring element. (I used 16x4 as structuring element)
Resultant Image after Step 5.
- Find external contours of this image.
- For each contour, find the
minAreaRect()
bounding it. - Select rectangles based on aspect ratio, minimum and maximum area, and angle with the horizontal. (I used 2.2 <= Aspect Ratio <= 8, 500 <= Area <=15000, and angle <= 45 degrees)
All
minAreaRect()
s are shown in orange and the one which satisfies our criteria is in green. - There may be false positives after this step, to filter it, use edge density. Edge Density is defined as the number of white pixels/total number of pixels in a rectangle. Set a threshold for edge density. (I used 0.5)
- Blur the detected regions.
You can apply other filters you deem suitable to increase recall and precision. The detection can also be trained using HOG+SVM to increase precision.
Abdul FatirAbdul Fatir4,31633 gold badges1818 silver badges4646 bronze badges
There is a new, open source library on GitHub that does ANPR for US and European plates. It looks pretty accurate and it should do exactly what you need (recognize the plate regions). Here is the GitHub project:https://github.com/openalpr/openalpr
Derrick JohnsonDerrick Johnson
I came across this one that is written in java javaANPR, I am looking for a c# library as well.
I would like a system where I can point a video camera at some sailing boats, all of which have large, identifiable numbers on them, and have it identify the boats and send a tweet when they sail past a video camera.
Bruce McLeodBruce McLeod
I have done some googling about this a couple of months ago. There are quite a few papers about this topic, but I never found any concrete open-source implementation. There are a lot of commercial implementations though, but none of them with a price quote, so they're probably pretty expensive.
Mauricio SchefferMauricio Scheffer88.3k1818 gold badges181181 silver badges270270 bronze badges
try this Simple Automatic Number Plate Recognition System
Open source and written with C#
Birol KuyumcuBirol Kuyumcu
Have a look at Java ANPR. Free license plate recognition...
Ralph WegnerRalph Wegner
Yes I use gocr at http://jocr.sourceforge.net/ its a commandline application which you could execute from your application. I use it in a couple of my applications.
keyokekeyoke
High performance ANPR Library - http://www.dtksoft.com/dtkanpr.php. This is commercial, but they provide trial key.
codegurucodeguru
http://licenseplate.sourceforge.net Python (I have not tested it)
SomatikSomatik
It maybe work looking at Character recoqnition software as there are many libraries out there that perform the same thing. I reading an image and storing it. Micrsoft office is able to read tiff files and return alphanumerics
MatthewMatthew
The blurring is not a problem but is there a library or component (open source preferred) that will help with finding a licence within a photo?
Ans: The CARMEN FreeFlow ANPR Software engine (Commerical)
abmvabmv4,3351313 gold badges5454 silver badges9898 bronze badges
Not the answer you're looking for? Browse other questions tagged imageocrcomputer-visionanpr or ask your own question.
Learn how to:
- Build the DLDT Inference Engine on your host Windows* machine.
- Use the engine to build the Security Camera Demo App from Open Model Zoo, also on your host Windows* machine.
- Run the Security Camera Demo App from inside a Windows* Docker container.
Step 1: Install Dependencies
To build the Security Barrier Camera Demo, you need to have these programs installed on your system. The versions for Visual Studio Community and the Intel® C++ Compiler are important!
NOTE: You must install Visual Studio Community before you install the Intel® C++ Compiler!
Program | Version | Use |
Visual Studio Community Visual C++ | 2017 (important) | building DLDT and security camera demos |
Intel® C++ Compiler | 19.0 (important) | building DLDT (absolutely necessary for certain builds to complete) |
Docker | 18.09.2 | building and running the Docker container |
CMake | 3.14.5 | generating VS 2017 solutions |
Git | 2.22.0.windows.1 | pulling from various repos |
Python 3 (not Python 2) | 3.6.8 | downloading open_model_zoo models |
GNU Wget | 1.11.4 | downloading various packages for CMake |
Visual Studio Community 2017
- Download the Visual Studio Installer.
- Only download the Desktop development with C++ workload.
Intel® C++ Compiler
- Download the Intel® Parallel Studio XE package installer. You will need to get a license for it (use the free trial or a student account). Make sure to select the customizable installer from the download page.
- https://software.intel.com/en-us/parallel-studio-xe/choose-download
- Intel® C++ Compiler 19.0 Update 4
- Intel® Threading Building Blocks 2019 Update 6
- NOTE: The Intel® Debugger Extension 8.2 may be automatically checked. You may leave it checked or uncheck it.
- On the top right corner, you may choose to uncheck the box next to IA-32 (it is not needed)
- If you follow these, your installation size (shown in the GUI) should be about 1.8GB.
- Select whether to consent to your information being collected.
- Next to IDE Integration, select Customize. Choose Visual Studio 2017 as your integration target.
- Proceed to install the packages you selected!
NOTE: The installation might take a while (~30 minutes to over 1 hour) with no progress update. Make sure to be patient and let the Finish button show up or else it will not install properly!
Docker
- Download Docker Desktop.
- After the installation completes, you will be prompted to log out.
- Log back in. On startup, Docker will ask you if you want to enable Hyper-V. Click Yes and you will be prompted to restart your computer. If you are never prompted to enable Hyper-V, see how to do so manually here.
- Right click Docker icon in the taskbar menu and select Switch to Windows* containers...
- Right click Docker icon, go to Settings
- In General, check 'Expose daemon on...'
- If you are working behind a proxy, in Proxies, check Manual proxy configuration and add your configurations.
- Make sure to click 'Apply' to properly apply these settings! You will have to restart Docker.
- You are now ready to use Docker!
Other
- Cmake Installer Page
- in the installer, make sure to check the option to add CMake to PATH
- Python 3.6.8
- in the installer, make sure to check the option to add Python to PATH
Step 2: Configure Proxies (only if needed)
If you are operating behind a proxy, make sure to set your HTTP_PROXY and HTTPS_PROXY environment variables to your corresponding proxy server addresses.
Step 3: Build the DLDT Inference Engine
NOTE: The steps to build the DLDT Inference Engine can also be found here.
First, clone dldt and its submodules:
NOTE: On rare occasion, git submodule init and git submodule update --recursive will fail to pull from the ade repository. Pay attention to possible error logs! You many need to check dldtinference-enginethirdpartyade to see if it is empty. If it is empty, you will need to manually clone ade from this repo.
Next, build the Visual Studio solution files. Pay attention to the parameters you pass into CMake! Note that specifying the Intel® C++ Compiler via the -T flag (as opposed to MSVC, etc.) is mandatory, or else many essential libraries (i.e. MKLDNNPlugin.dll) will fail to build!
- Make sure to change the Intel® C++ Compiler version in your command syntax to match whichever version you have installed (if you are not using version 19.0)
- Make sure to specify the Win64 Visual Studio generator, or else the process will silently fail.
NOTE: If this fails, try running this command on the 'Intel Compiler 19.0 Update 4 Intel(R) 64 Visual Studio 2017' shell environment. It should be located at C:ProgramDataMicrosoftWindowsStart MenuProgramsIntel Parallel Studio XE 2019Compiler and Performance LibrariesCommand Prompt with Intel® Compiler 19.0 Update 4.
Finally, build the executables:
NOTE: You MUST build in 'Release' mode! Failing to build in 'Release' mode will cause the Security Barrier Camera Demo to fail to build later on (it will not recognize the Directory name or the names of several .dll's)
Step 3: Build OpenCV
First, clone opencv.
Next, build the OpenCV solution files, and then OpenCV!
NOTE: The DLDT repository actually contains all the necessary OpenCV libraries for building the camera demo. However, building the app using the original repository solves some problems with decoding .mp4 videos inside the Windows* Docker container.
Step 4: Build Security Camera Barrier Demo from Open Model Zoo
NOTE: The steps to build the demo can also be found here.
First, clone the Open Model Zoo repository and switch to the 2019 branch.
We want to build the demos manually using the Inference Engine libraries from dldt, so we set the InferenceEngine_DIR environment variable to the dldt build directory (your path depends on where your dldt folder is located):
We want to use the OpenCV libraries from our own installation of OpenCV, not the libraries included in the dldt repo), so we will set the OpenCV_DIR environment variable to the directory containing the OpenCVConfig.cmake file.
NOTE: (See note in Step 3) Alternatively, you may link the OpenCV directory inside the dldt repo:
The benefit of this is that you don't actually need to install OpenCV from its own repo (you can skip step 3). However, if you do this, the demo application will not be able to read .mp4 files.
Next, we simply run
to build the demos. The demos, including security_barrier_camera_demo.exe, are found in C:Users<username>DocumentsIntelOpenVINOomz_demos_build_buildintel64Release , where <username> is your account username.
Download Models
In order to download the appropriate models for the security barrier camera demo, we run these commands:
The downloader.py script downloads these three models, all of which will be used by security_barrier_camera_demo.exe:
- vehicle-license-plate-detection-barrier-0106
- vehicle-attributes-recognition-barrier-0039
- license-plate-recognition-barrier-0001
A full list of models can be found here. However, these are the only models you should download for the security barrier camera app demo.
At this point, everything we need is built, and all we need to do is transfer necessary files to the Docker container.
Step 5: Aggregating Dynamic Libraries
In order for security_camera_barrier_demo.exe to run, it must have access to the .dll's below.
NOTE: The IntelSWTools libraries come with installing the Intel® C++ Compiler.
File Directory | Library Files |
dldtinference-enginebinintel64Release(DLDT) | cpu_extension.dll format_reader.dll inference_engine.dll MKLDNNPlugin.dll |
C:Program Files (x86)IntelSWTools...* | libmmd.dll svml_dispmd.dll |
opencvbuildbinRelease (OpenCV) | opencv_core411.dll opencv_highgui411.dll opencv_imgproc411.dll opencv_imgcodecs411.dll opencv_videoio411.dll |
opencvbuild3rdpartyffmpeg (OpenCV) | opencv_videoio_ffmpeg411_64.dll |
dldtinference-enginetemptbblib (DLDT) | tbb.dll tbb_debug.dll tbbmalloc.dll tbbmalloc_debug.dll |
*...IntelSWToolscompilers_and_libraries_2019windowsredistintel64_wincompiler
We also need some .dll's that come with the Visual C++ runtime, which are found in System32 on your host. You do not need to copy these files into the Docker container, however, as we can install the Visual C++ runtime via Dockerfile.
File Directory | Library Files |
C:WindowsSystem32 | vcruntime140.dll msvcp140.dll concrt140.dll |
Make these .dll's accessible by either placing them in the same folder as the executable, or adding the directories containing them to your PATH.
Step 6: Building the Docker Image
Before we run docker build, we need to arrange all the necessary files in a special format so that we can copy them into the Docker container.
Here is a file structure guide and a list of files that must be put inside the Docker container.
File Structure
List of Files
- security_barrier_camera_demo.exe
- Found in ...DocumentsIntelOpenVINOomz_demos_buildintel64Release
- This is the security camera demo app
- vc_redist.x64.exe
- You may download it from here.
- This executable will be used to install the Visual C++ runtime.
- All of the .dll files listed above (in the table)
- Comprehensive list:
- cpu_extension.dll
- format_reader.dll
- inference_engine.dll
- MKLDNNPlugin.dll
- libmmd.dll
- svml_dispmd.dll
- opencv_core411.dll
- opencv_highgui411.dll
- opencv_imgproc411.dll
- opencv_imgcodecs411.dll
- opencv_videoio411.dll
- opencv_videoio_ffmpeg411_64.dll
- tbb.dll
- tbb_debug.dll
- tbbmalloc.dll
- tbbmalloc_debug.dll
- Comprehensive list:
- Compressed machine learning models from Open Model Zoo (both the .bin and .xml files for each)
- License Plate Recognition Barrier Model
- Found in open_model_zoomodel_downloaderSecurityobject_attributesvehicleresnet10_update_1dldt
- Vehicle Attributes Recognition Barrier Model
- Found in open_model_zoomodel_downloaderSecurityobject_detectionbarrier0106dldt
- Vehicle License Plate Detection Barrier Model
- Found in open_model_zoomodel_downloaderSecurityoptical_character_recognitionlicense_platedldt
- License Plate Recognition Barrier Model
Docker build
Go to the directory containing your Dockerfile and build the image.
NOTE: Pulling the Windows* amd-64 image may take a LONG time, and at times you may see full progress bars with no progress update (during 'Extracting' phase) for over 20 minutes. Be patient! Check your task manager, and if Docker is still running, it means that the build is probably working, even if it may not seem so.
Dockerfile
NOTE: If you see errors on build such as 'a Windows version XXX-based image is incompatible with XXX host,' check your Windows* version. TheThe provided Dockerfile uses the Windows* 10 1903 base image, but an older Windows* host cannot run a newer Windows* container. For example, if your Windows* Version is Windows* 10 1809, you will need to edit the base image to mcr.microsoft.com/windows:1809-amd64
Step 7: Running in Docker Container
Use the command below to run the camera demo in the Docker container.
- [$path] should be the absolute path to the folder containing the images / videos you want to run the app on.
- i.e. you may download this image of a car, and if the image's full path is '/home/folder/car_1.bmp', replace [$path] with '/home/folder'.
- The -no_show and -r flags are used to make sure that the application output is printed as text (as we cannot view image or video output from the Docker container).
If everything works properly, the output should print some inference results describing the car and its license plate.
For more complete information about compiler optimizations, see our Optimization Notice.