CN108197619A - A kind of localization method based on signboard image, device, equipment and storage medium - Google Patents
A kind of localization method based on signboard image, device, equipment and storage medium Download PDFInfo
- Publication number
- CN108197619A CN108197619A CN201711447370.XA CN201711447370A CN108197619A CN 108197619 A CN108197619 A CN 108197619A CN 201711447370 A CN201711447370 A CN 201711447370A CN 108197619 A CN108197619 A CN 108197619A
- Authority
- CN
- China
- Prior art keywords
- signboard
- image
- present image
- candidate
- present
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a kind of localization method based on signboard image, device, equipment and storage mediums.The method includes：Camera site according to present image is screened from signboard library obtains candidate signboard；The signboard image section included in the present image with screening obtained candidate signboard is matched, obtains matched candidate signboard；Using the location information of matched candidate signboard as current location information.Signboard image and the location information rough with user in technical solution extraction image provided in an embodiment of the present invention are combined obtains accurate anchor point by training matching, and it is not influenced by factors such as weather and positions, can realize high accuracy positioning and relatively easily carries out data maintenance.
Description
Technical field
The present embodiments relate to field of locating technology more particularly to a kind of localization method based on signboard image, device,
Equipment and storage medium.
Background technology
With the fast development of location technology and the continuous renewal of intelligent terminal technology, location based service extensive use
In the every field of Working Life.Requirement of the people to positioning accuracy is higher and higher, and especially in terms of positioning indoors, precision is asked
Topic is always difficult point.Location technology is based on premised on location-based service.
At present, outdoor positioning is based primarily upon global positioning system (Global Positioning System, GPS), indoor
Positioning is based primarily upon WIFI, bluetooth and earth magnetism.Above-mentioned several location technology usage scenarios are different, caused problem and shortage
It is different, such as：The application scenarios of GPS can only be limited in outdoor vacant lot and the preferable situation of weather, located sites；WIFI and indigo plant
The difficult point of tooth positioning is to obtain the WIFI of high quality and bluetooth hot spot data, if the quality of WIFI and bluetooth hot spot data is not
Height, then positioning accuracy is impacted, and the position of WIFI and bluetooth hot spot may change frequently, data maintenance it is difficult；Ground
The difficult point of magnetic orientation is that data acquire, and the variation of earth magnetism situation is faster, data maintenance difficulty bigger.
Invention content
The embodiment of the present invention provides a kind of localization method based on signboard image, device, equipment and storage medium, Ke Yishi
Show high accuracy positioning and relatively easily carry out data maintenance.
In a first aspect, an embodiment of the present invention provides a kind of localization method based on signboard image, including：
Camera site according to present image is screened from signboard library obtains candidate signboard；
The signboard image section included in the present image with screening obtained candidate signboard is matched, is obtained
The candidate signboard matched；
Using the location information of matched candidate signboard as current location information.
Second aspect, the embodiment of the present invention additionally provide a kind of localization method based on signboard image, including：
Obtain the present image of user's shooting and the camera site of the present image；
According to the camera site of the present image and the present image, Location Request, wherein institute are sent to server-side
It states Location Request and is used to indicate the following operation of server-side execution：Camera site according to the present image is from signboard library
Screening obtains candidate signboard；The candidate signboard progress that the signboard image section included in the present image is obtained with screening
Match, obtain matched candidate signboard；Using the location information of matched candidate signboard as current location information；
Receive the current location information that the server-side returns.
The third aspect, the embodiment of the present invention additionally provide a kind of positioning device based on signboard image, which includes：
Screening module obtains candidate signboard for being screened from signboard library according to the camera site of present image；
Matching module, for by the signboard included in the present image image section and the obtained candidate signboard of screening into
Row matching obtains matched candidate signboard；
Locating module, for using the location information of matched candidate signboard as current location information.
Fourth aspect, the embodiment of the present invention additionally provide a kind of positioning device based on signboard image, which includes：
Acquisition module, for obtaining the camera site of the present image of user's shooting and the present image；
Location Request sending module, for the camera site according to the present image and the present image, to service
End sends Location Request, wherein the Location Request, which is used to indicate the server-side, performs following operation：According to the current figure
It is screened from signboard library and obtains candidate signboard in the camera site of picture；By the signboard image section included in the present image and sieve
Obtained candidate signboard is selected to be matched, obtains matched candidate signboard；Location information using matched candidate signboard is as ought
Front position information；
Receiving module, for receiving the current location information that the server-side returns.
5th aspect, the embodiment of the present invention additionally provide a kind of equipment, and the equipment includes：
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are performed by one or more of processors so that one or more of processing
Device realizes the localization method as described above based on signboard image.
6th aspect, the embodiment of the present invention additionally provide a kind of computer readable storage medium, are stored thereon with computer
Program, which is characterized in that the localization method as described above based on signboard image is realized when the program is executed by processor.
The embodiment of the present invention screens from signboard library according to the camera site of present image and obtains candidate signboard, and will be current
The signboard image section included in image is matched with screening obtained candidate signboard, obtains matched candidate signboard, general
The location information of candidate signboard matched is as current location information.In technical solution extraction image provided in an embodiment of the present invention
Signboard image and the location information rough with user are combined obtains accurate anchor point, and not by weather and position by training matching
It the influence of factors such as puts, can realize high accuracy positioning and relatively easily carry out data maintenance.
Description of the drawings
Fig. 1 is the flow chart of the localization method based on signboard image in the embodiment of the present invention one；
Fig. 2 is the flow chart of the localization method based on signboard image in the embodiment of the present invention two；
Fig. 3 is the acquisition schematic diagram in the signboard library in the embodiment of the present invention two；
Fig. 4 is the flow chart of the localization method based on signboard image in the embodiment of the present invention three；
Fig. 5 is another flow chart of the localization method based on signboard image in the embodiment of the present invention three；
Fig. 6 is the structure diagram of the positioning device based on signboard image in the embodiment of the present invention four；
Fig. 7 is the structure diagram of the positioning device based on signboard image in the embodiment of the present invention five；
Fig. 8 is the structure diagram of the equipment in the embodiment of the present invention six.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limitation of the invention.It also should be noted that in order to just
Part related to the present invention rather than entire infrastructure are illustrated only in description, attached drawing.
Embodiment one
Fig. 1 is the flow chart of the localization method based on signboard image in the embodiment of the present invention one, and the present embodiment is applicable
In the situation of the positioning based on signboard image, this method can be performed by the positioning device based on signboard image, the present embodiment
Executive agent can be server-side, this method can specifically include：
S110, the camera site according to present image screen from signboard library and obtain candidate signboard.
Wherein, the present image can be the image that user uses filming apparatus (such as camera or mobile phone) shooting, described
The camera site of present image can be the GPS coordinate of user's current coarse positioning.
The signboard is primarily used to the title and mark in instruction shop, can be described as shop mark (title in shop), signboard exists
Trading company is legally defined as, trading company can be made of word and pattern etc..Signboard is not equal to other any form publications and puts up
Ad content and advertisement, signboard can have perpendicular trick, horizontal trick or the horizontal autograph number or be suspended under eaves huge on memorial archway in front of the door
Word is laterally set on building by plaque.
The signboard library can be all points of interest (Point of Interest, the POI) figure for being stored in advance in server-side
As the signboard data in data, the signboard data can be based on structuring or unstructured database is stored, each to recruit
Board data correspond to a data library record, and the data-base recording stores the corresponding POI details of the signboard.
Wherein, the point of interest is the term managed in information system, and it is right to refer to the geography that all can be abstracted as a little
As closely related geographical entity of more especially living with people, such as school, bank, restaurant, gas station, hospital and supermarket
Deng.Point of interest is mainly used for that the address of things or event is described, and can largely enhance to things or thing
The descriptive power and query capability of part position improve the accuracy and speed of geo-location.
Specifically, the shooting rough position according to present image, can filter out from signboard library near current location
The signboard of pre-set quantity can specifically be set as candidate signboard, the pre-set quantity according to actual conditions,
Such as：In urban district, signboard quantity is more, quantity can set it is larger, can be 20；In suburb, signboard negligible amounts, quantity can
It is smaller to set, can be 5.
S120, the signboard image section included in the present image is matched with screening obtained candidate signboard,
Obtain matched candidate signboard.
Wherein, the signboard image section can be the signboard image that includes of present image, and the present image can be with
1 signboard image is only included, multiple signboard images can also be included.
Specifically, the signboard optical sieving in the present image can be come out, and according to signboard by deep learning
The feature of image can be matched with the candidate signboard screened in S110, according to matching result, by most like candidate
Signboard is as the signboard matched.
S130, using the location information of matched candidate signboard as current location information.
Specifically, the location information of the matched candidate signboard that can store server-side is as current location information
It is sent to user.
The present embodiment screens from signboard library according to the fuzzy camera site of present image and obtains candidate signboard, and by institute
It states the signboard image section included in present image to be matched with screening obtained candidate signboard, obtains matched candidate trick
Board, the location information of matched candidate's signboard is current location information.Since the location information in signboard library is accurate, carry
It takes the signboard image in image and the location information rough with user is combined by training matching that can obtain accurate anchor point,
And it is not influenced by factors such as weather and positions, can realize high accuracy positioning and relatively easily carries out data maintenance.
Embodiment two
Fig. 2 is the flow chart of the localization method based on signboard image in the embodiment of the present invention two.The present embodiment is being implemented
On the basis of example one, the above-mentioned localization method based on signboard image has been advanced optimized.Correspondingly, the method for the present embodiment is specific
It can include：
S201, the signboard size in each signboard sample image is clustered, obtains at least one general signboard size.
Wherein, the signboard sample image can be the signboard figure of a certain number of different signboard sizes of manual confirmation
Picture.
It is clustered specifically, k-means algorithms may be used to the signboard size in each signboard sample image, it will be each
Signboard sample image is classified according to the difference of signboard size, obtains at least one classification, counts the possibility ruler of signboard image
It is very little, it can be using the signboard size of the classification as general signboard size.Illustratively, if being obtained after each signboard sample image cluster
Two classification that signboard size is 3 × 11 and signboard size is 4 × 8, then obtain two general signboard sizes, and respectively 3 × 11
With 4 × 8.
S202, using at least one general signboard size as the anchor point size of Faster-RCNN convolutional neural networks.
Wherein, the anchor point can be used for describing the size of object, anchor point and Faster-RCNN (Faster- in advance
Region Convolutional Neural Networks) convolutional neural networks eigenmatrix it is corresponding, anchor point size can
Think the size for finely tuning frame during determining signboard position.
Specifically, at least one general signboard size that can determine S201 is as Faster-RCNN convolutional Neural nets
The anchor point size of network.
S203, the signboard detection model is built based on Faster-RCNN convolutional neural networks, wherein the Faster-
The convolution kernel of RCNN convolutional neural networks is rectangle convolution kernel.
Wherein it is possible to the conventional square convolution kernel of the Faster-RCNN convolutional neural networks is replaced with into rectangle
Convolution kernel goes the scene of adaptation signboard, such as can conventional 3 × 3 square convolution kernels be replaced with 3 × 11 or 11 × 3 etc.
Strip convolution kernel.Since signboard is more strip, after convolution kernel is replaced, it can preferably retain the spy of signboard entirety
Sign, and it is obviously improved accuracy rate.
Specifically, using the part signboard data manually demarcated in POI image datas as Faster-RCNN convolutional Neurals
The input of network is trained, and obtains signboard detection model.
S204, the Location Request that client is sent is received, wherein the Location Request includes the present image and institute
State the camera site of present image.
Specifically, server-side can receive the Location Request that user sends in client, wherein being wrapped in the Location Request
The camera site of present image and present image is included, the camera site can be the GPS coordinate of user's current coarse positioning.
S205, the present image is detected based on signboard detection model, obtains what is included in the present image
Signboard image section.
In the present embodiment, the signboard detection model can be preset in server-side, can be using present image as trick
The input of board detection model is trained, and output is the parallel most with x, y-axis of all signboard image sections in present image
Small boundary rectangle, the size of the minimum enclosed rectangle can be anchor point size.Since the number of anchor point size increases so that inspection
The accuracy rate of signboard image section included in present image is surveyed to be greatly improved.
S206, the camera site according to present image screen from signboard library and obtain candidate signboard.
Fig. 3 is the acquisition schematic diagram in the signboard library in the embodiment of the present invention two.Referring to Fig. 3, the POI pictures that will manually demarcate
Signboard data in data are trained as the input of Faster-RCNN convolutional neural networks, obtain signboard detection model.It will
The accurate signboard data of training are stored in signboard library, the signboard data can be based on structuring or unstructured database into
Row storage, each signboard data correspond to a data library record, and it is detailed that the data-base recording stores the corresponding POI of the signboard
Information.
S207, image characteristics extraction is carried out, and tie according to extraction to the signboard image section included in the present image
Fruit obtains the feature vector of the signboard image section.
Specifically, the signboard image section included in the present image can be led to as the input of signboard detection model
Cross the last several layers of progress Scale invariant features transforms of re -training (Scale Invariant Feature Transform,
SIFT) image characteristics extraction, and can obtain the feature vector of the signboard image section according to result is extracted.
Need what is illustrated, signboard data also pass through the training of signboard detection model in the signboard library in the S206, therefore
Each the feature based on SIFT image characteristics extractions can also be stored in the corresponding data-base recording of signboard data.
S208, the feature vector of the signboard image section and the feature vector of each candidate signboard being previously obtained are carried out
Matching.
S209, foundation matching result are chosen and the most matched candidate signboard of the signboard image section.
S210, using the location information of matched candidate signboard as current location information.
The present embodiment is by predefining the anchor point size of Faster-RCNN convolutional neural networks and based on Faster-
RCNN convolutional neural networks build signboard detection model, receive the Location Request that client is sent, and according to the bat of present image
It acts as regent to put to screen from signboard library and obtains candidate signboard；It is trained, obtains using present image as the input of signboard detection model
The signboard image section included into present image carries out feature extraction；It and will be by the feature vector of the signboard image section
It is matched with the feature vector of each candidate signboard being previously obtained, using the location information of most matched candidate signboard as current
Location information.Due to the increase of the number of the anchor point size of Faster-RCNN convolutional neural networks and rectangle convolution kernel so that instruction
Practice the size for being more adapted to signboard, can preferably retain the feature of signboard entirety, recall rate and accuracy rate are greatly improved；And it recruits
Location information in board library is accurate, and the signboard image and the location information rough with user extracted in image is combined and pass through instruction
Practice matching can obtain accurate anchor point, and do not influenced by factors such as weather and positions, can realize high accuracy positioning and
Relatively easily carry out data maintenance.
Embodiment three
Fig. 4 is the flow chart of the localization method based on signboard image in the embodiment of the present invention three, and the present embodiment is applicable
In the situation of the positioning based on signboard image, this method can be performed by the positioning device based on signboard image, the present embodiment
Executive agent can be client, this method can specifically include：
The camera site of S310, the present image for obtaining user's shooting and the present image.
Wherein, the present image can use the image of client (such as mobile phone) shooting, the present image for user
Camera site can be user's current coarse positioning GPS coordinate.
Specifically, obtain the present image of user's shooting and the rough position of present image.
S320, the camera site according to the present image and the present image send Location Request to server-side,
Described in Location Request be used to indicate the server-side and perform following operation：Camera site according to the present image is from signboard
Screening obtains candidate signboard in library；By the signboard image section included in the present image and the obtained candidate signboard of screening into
Row matching obtains matched candidate signboard；Using the location information of matched candidate signboard as current location information.
Specifically, client can be detected the present image based on preset signboard detection model, institute is obtained
State the signboard image section included in present image；Include the signboard image portion in the present image to server-side transmission
Divide the Location Request with the camera site of the present image.It, can be with since the signboard detection model is preset in the client
Reduce the web database technology that client is transmitted to server-side.
Wherein described Location Request is used to indicate the server-side and performs following operation：Shooting according to the present image
It is screened from signboard library and obtains candidate signboard in position；The signboard image section included in the present image is obtained with screening
Candidate signboard is matched, and obtains matched candidate signboard；Using the location information of matched candidate signboard as present bit confidence
Breath.
S330, the current location information that the server-side returns is received.
Fig. 5 is another flow chart of the localization method based on signboard image in the embodiment of the present invention three, illustrates visitor
Family end and interactive process of server-side when based on signboard framing.Referring to Fig. 5, in the client, open camera and carry out
Shooting, obtains present image, and present image is trained based on preset signboard detector in the client, extracts signboard
Data, and signboard data and coarse GPS information are sent to server-side.In server-side, the signboard number that client is sent is received
According to coarse GPS information, neighbouring signboard is screened from signboard library according to coarse GPS information, and find with SIFT feature attached
Most matched signboard near signboard, it is the elements of a fix to obtain the most corresponding POI coordinates of matching signboard.
For the present embodiment by obtaining the present image of user's shooting and the camera site of the present image, client can
To obtain the signboard image section included in present image based on preset signboard detection model, according to the present image and institute
The camera site of present image is stated, Location Request is sent to server-side, wherein the Location Request is used to indicate server-side by institute
It states the signboard image section included in present image to be matched with screening obtained candidate signboard, obtains matched candidate signboard
Using location information as current location information.The technical solution of the present embodiment is preset at client due to the signboard detection model
In end, it is possible to reduce the web database technology that client is transmitted to server-side, and the location information in signboard library is accurate, extraction
Signboard image and the location information rough with user in image are combined can obtain accurate anchor point by training matching, and
It is not influenced by factors such as weather and positions, can realize high accuracy positioning and relatively easily carries out data maintenance.
Example IV
Fig. 6 is the structure diagram of the positioning device based on signboard image in the embodiment of the present invention four, and described device can
To include：
Screening module 410 obtains candidate signboard for being screened from signboard library according to the camera site of present image；
Matching module 420, for the candidate trick for obtaining the signboard included in the present image image section with screening
Board is matched, and obtains matched candidate signboard；
Locating module 430, for using the location information of matched candidate signboard as current location information.
Further, the matching module 420 specifically can be used for：
Image characteristics extraction is carried out to the signboard image section included in the present image, and is obtained according to extraction result
The feature vector of the signboard image section；
The feature vector of the signboard image section is matched with the feature vector of each candidate signboard being previously obtained；
According to matching result, choose and the most matched candidate signboard of the signboard image section.
Further, which can also include the first Location Request module, be specifically used for：
The Location Request that client is sent is received, wherein the Location Request includes the signboard figure in the present image
As part and the camera site of the present image, the signboard image section is that the client is detected based on preset signboard
Model is detected the present image.
Further, which can also include the second Location Request module, be specifically used for：
Receive client send Location Request, wherein the Location Request include the present image and it is described currently
The camera site of image；
The present image is detected based on signboard detection model, obtains the signboard figure included in the present image
As part.
Further, which can also include signboard detection model structure module, be specifically used for：
The signboard detection model is built based on Faster-RCNN convolutional neural networks, wherein described Faster-RCNN volumes
The convolution kernel of product neural network is rectangle convolution kernel.
Further, which can also include anchor module, be specifically used for：
Signboard size in each signboard sample image is clustered, obtains at least one general signboard size；
Using at least one general signboard size as the anchor point size of the Faster-RCNN convolutional neural networks.
The positioning device based on signboard image that the embodiment of the present invention is provided can perform the embodiment of the present invention one and/or reality
The localization method based on signboard image that example two is provided is applied, has the corresponding function module of execution method and advantageous effect.
Embodiment five
Fig. 7 is the structure diagram of the positioning device based on signboard image in the embodiment of the present invention five, and described device can
To include：
Acquisition module 510, for obtaining the camera site of the present image of user's shooting and the present image；
Location Request sending module 520, for the camera site according to the present image and the present image, to clothes
Business end sends Location Request, wherein the Location Request, which is used to indicate the server-side, performs following operation：According to described current
It is screened from signboard library and obtains candidate signboard in the camera site of image；By the signboard image section included in the present image with
It screens obtained candidate signboard to be matched, obtains matched candidate signboard；Using the location information of matched candidate signboard as
Current location information；
Receiving module 530, for receiving the current location information that the server-side returns.
Further, the Location Request sending module 520 specifically can be used for：
The present image is detected based on preset signboard detection model, obtains what is included in the present image
Signboard image section；
Include the shooting of signboard image section and the present image in the present image to server-side transmission
The Location Request of position.
The positioning device based on signboard image that the embodiment of the present invention is provided can perform the embodiment of the present invention three and be provided
The localization method based on signboard image, have the corresponding function module of execution method and advantageous effect.
Embodiment six
Fig. 8 is the structure diagram of the equipment in the embodiment of the present invention six.Fig. 8 shows of the invention real suitable for being used for realizing
Apply the block diagram of the exemplary servers 612 of mode.The server 612 that Fig. 8 is shown is only an example, should not be to of the invention real
The function and use scope for applying example bring any restrictions.
As shown in figure 8, server 612 is showed in the form of universal computing device.The component of server 612 can include but
It is not limited to：One or more processor 616, system storage 628, connection different system component is (including system storage 628
With processor 616) bus 618.
Bus 618 represents one or more in a few class bus structures, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor 616 or total using the local of the arbitrary bus structures in a variety of bus structures
Line.For example, these architectures include but not limited to industry standard architecture (ISA) bus, microchannel architecture
(MAC) bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) are total
Line.
Server 612 typically comprises a variety of computer system readable media.These media can be it is any being capable of bedding and clothing
The usable medium that business device 612 accesses, including volatile and non-volatile medium, moveable and immovable medium.
System storage 628 can include the computer system readable media of form of volatile memory, such as deposit at random
Access to memory (RAM) 630 and/or cache memory 632.Server 612 may further include it is other it is removable/can not
Mobile, volatile/non-volatile computer system storage medium.Only as an example, storage system 634 can be used for read-write not
Movably, non-volatile magnetic media (Fig. 8 do not show, commonly referred to as " hard disk drive ").Although being not shown in Fig. 8, can with
It provides for moving the disc driver of non-volatile magnetic disk (such as " floppy disk ") read-write and to removable non-volatile
The CD drive of CD (such as CD-ROM, DVD-ROM or other optical mediums) read-write.In these cases, each driving
Device can be connected by one or more data media interfaces with bus 618.Memory 628 can include at least one program
Product, the program product have one group of (for example, at least one) program module, these program modules are configured to perform the present invention
The function of each embodiment.
Program/utility 640 with one group of (at least one) program module 642, can be stored in such as memory
In 628, such program module 642 includes but not limited to operating system, one or more application program, other program modules
And program data, the realization of network environment may be included in each or certain combination in these examples.Program module 642
Usually perform the function and/or method in embodiment described in the invention.
Server 612 can also be with one or more external equipments 614 (such as keyboard, sensing equipment, display 624 etc.)
Communication can also enable a user to the equipment interacted with the server 612 communication and/or with causing the clothes with one or more
Any equipment (such as network interface card, modem etc.) that business device 612 can communicate with one or more of the other computing device
Communication.This communication can be carried out by input/output (I/O) interface 622.Also, server 612 can also be fitted by network
Orchestration 620 and one or more network (such as LAN (LAN), wide area network (WAN) and/or public network, such as because of spy
Net) communication.As shown in the figure, network adapter 620 is communicated by bus 618 with other modules of server 612.It should be understood that
Although not shown in the drawings, can combine server 612 uses other hardware and/or software module, including but not limited to：Micro- generation
Code, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and data backup are deposited
Storage system etc..
Processor 616 is stored in program in system storage 628 by operation, so as to perform various functions application and
Data processing, such as realize the localization method based on signboard image that the embodiment of the present invention is provided, this method includes：
Camera site according to present image is screened from signboard library obtains candidate signboard；
The signboard image section included in the present image with screening obtained candidate signboard is matched, is obtained
The candidate signboard matched；
Using the location information of matched candidate signboard as current location information.
Embodiment seven
The embodiment of the present invention seven additionally provides a kind of computer readable storage medium, is stored thereon with computer program, should
The localization method based on signboard image provided such as the embodiment of the present invention, this method packet are provided when program is executed by processor
It includes：
Camera site according to present image is screened from signboard library obtains candidate signboard；
The signboard image section included in the present image with screening obtained candidate signboard is matched, is obtained
The candidate signboard matched；
Using the location information of matched candidate signboard as current location information.
The arbitrary of one or more computer-readable media may be used in the computer storage media of the embodiment of the present invention
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device or arbitrary above combination.The more specific example (non exhaustive list) of computer readable storage medium includes：Tool
There are one or the electrical connections of multiple conducting wires, portable computer diskette, hard disk, random access memory (RAM), read-only memory
(ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer-readable storage
Medium can be any tangible medium for including or storing program, which can be commanded execution system, device or device
Using or it is in connection.
Computer-readable signal media can include in a base band or as a carrier wave part propagation data-signal,
Wherein carry computer-readable program code.Diversified forms may be used in the data-signal of this propagation, including but it is unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By instruction execution system, device either device use or program in connection.
The program code included on computer-readable medium can be transmitted with any appropriate medium, including --- but it is unlimited
In wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
It can write to perform the computer that operates of the present invention with one or more programming language or combinations
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
Further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully performs, partly perform on the user computer on the user computer, the software package independent as one performs, portion
Divide and partly perform or perform on a remote computer or server completely on the remote computer on the user computer.
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including LAN (LAN) or
Wide area network (WAN)-be connected to subscriber computer or, it may be connected to outer computer (such as is carried using Internet service
Pass through Internet connection for quotient).
Note that it above are only presently preferred embodiments of the present invention and institute's application technology principle.It will be appreciated by those skilled in the art that
The present invention is not limited to specific embodiment described here, can carry out for a person skilled in the art various apparent variations,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above example to the present invention
It is described in further detail, but the present invention is not limited only to above example, without departing from the inventive concept, also
It can include other more equivalent embodiments, and the scope of the present invention is determined by scope of the appended claims.
Claims (18)
1. a kind of localization method based on signboard image, which is characterized in that including：
Camera site according to present image is screened from signboard library obtains candidate signboard；
The signboard image section included in the present image with screening obtained candidate signboard is matched, is obtained matched
Candidate signboard；
Using the location information of matched candidate signboard as current location information.
2. according to the method described in claim 1, it is characterized in that, by the signboard image section included in the present image with
It screens obtained candidate signboard to be matched, obtains most matched candidate signboard, including：
Image characteristics extraction is carried out to the signboard image section that is included in the present image, and is obtained according to extraction result described
The feature vector of signboard image section；
The feature vector of the signboard image section is matched with the feature vector of each candidate signboard being previously obtained；
According to matching result, choose and the most matched candidate signboard of the signboard image section.
3. according to the method described in claim 1, it is characterized in that, the camera site according to present image is from signboard library
Before screening obtains candidate signboard, including：
The Location Request that client is sent is received, wherein the Location Request includes the signboard image portion in the present image
Divide the camera site with the present image, the signboard image section is the client based on preset signboard detection model
The present image is detected.
4. according to the method described in claim 1, it is characterized in that, the camera site according to present image is from signboard library
Before screening obtains candidate signboard, further include：
The Location Request that client is sent is received, wherein the Location Request includes the present image and the present image
Camera site；
The present image is detected based on signboard detection model, obtains the signboard image portion included in the present image
Point.
5. according to the method described in claim 3 or claim 4, which is characterized in that the structure of signboard detection model includes：
The signboard detection model is built based on Faster-RCNN convolutional neural networks, wherein the Faster-RCNN convolution is refreshing
Convolution kernel through network is rectangle convolution kernel.
6. according to the method described in claim 5, it is characterized in that, described built based on Faster-RCNN convolutional neural networks
Before the signboard detection model, including：
Signboard size in each signboard sample image is clustered, obtains at least one general signboard size；
Using at least one general signboard size as the anchor point size of the Faster-RCNN convolutional neural networks.
7. a kind of localization method based on signboard image, which is characterized in that including：
Obtain the present image of user's shooting and the camera site of the present image；
According to the camera site of the present image and the present image, Location Request is sent to server-side, wherein described fixed
Position request is used to indicate the server-side and performs following operation：It is screened from signboard library camera site according to the present image
Obtain candidate signboard；The signboard image section included in the present image is matched with screening obtained candidate signboard,
Obtain matched candidate signboard；Using the location information of matched candidate signboard as current location information；
Receive the current location information that the server-side returns.
8. the method according to the description of claim 7 is characterized in that shooting according to the present image and the present image
Position sends Location Request to server-side, including：
The present image is detected based on preset signboard detection model, obtains the signboard included in the present image
Image section；
Include the camera site of signboard image section and the present image in the present image to server-side transmission
Location Request.
9. a kind of positioning device based on signboard image, which is characterized in that including：
Screening module obtains candidate signboard for being screened from signboard library according to the camera site of present image；
Matching module, for the candidate signboard progress for obtaining the signboard included in the present image image section with screening
Match, obtain matched candidate signboard；
Locating module, for using the location information of matched candidate signboard as current location information.
10. device according to claim 9, which is characterized in that the matching module is specifically used for：
Image characteristics extraction is carried out to the signboard image section that is included in the present image, and is obtained according to extraction result described
The feature vector of signboard image section；
The feature vector of the signboard image section is matched with the feature vector of each candidate signboard being previously obtained；
According to matching result, choose and the most matched candidate signboard of the signboard image section.
11. device according to claim 9, which is characterized in that further include the first Location Request module, be specifically used for：
The Location Request that client is sent is received, wherein the Location Request includes the signboard image portion in the present image
Divide the camera site with the present image, the signboard image section is the client based on preset signboard detection model
The present image is detected.
12. device according to claim 9, which is characterized in that further include the second Location Request module, be specifically used for：
The Location Request that client is sent is received, wherein the Location Request includes the present image and the present image
Camera site；
The present image is detected based on signboard detection model, obtains the signboard image portion included in the present image
Point.
13. according to the device described in claim 11 or claim 12, which is characterized in that further include signboard detection model structure
Module is specifically used for：
The signboard detection model is built based on Faster-RCNN convolutional neural networks, wherein the Faster-RCNN convolution is refreshing
Convolution kernel through network is rectangle convolution kernel.
14. device according to claim 13, which is characterized in that further include anchor module, be specifically used for：
Signboard size in each signboard sample image is clustered, obtains at least one general signboard size；
Using at least one general signboard size as the anchor point size of the Faster-RCNN convolutional neural networks.
15. a kind of positioning device based on signboard image, which is characterized in that including：
Acquisition module, for obtaining the camera site of the present image of user's shooting and the present image；
Location Request sending module for the camera site according to the present image and the present image, is sent out to server-side
Location Request is sent, wherein the Location Request, which is used to indicate the server-side, performs following operation：According to the present image
It is screened from signboard library and obtains candidate signboard in camera site；By the signboard image section included in the present image and screen
To candidate signboard matched, obtain matched candidate signboard；Using the location information of matched candidate signboard as present bit
Confidence ceases；
Receiving module, for receiving the current location information that the server-side returns.
16. device according to claim 15, which is characterized in that the Location Request sending module is specifically used for：
The present image is detected based on preset signboard detection model, obtains the signboard included in the present image
Image section；
Include the camera site of signboard image section and the present image in the present image to server-side transmission
Location Request.
17. a kind of equipment, which is characterized in that the equipment includes：
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are performed by one or more of processors so that one or more of processors are real
Now such as the localization method according to any one of claims 1-8 based on signboard image.
18. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
Such as the localization method according to any one of claims 1-8 based on signboard image is realized during execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711447370.XA CN108197619A (en) | 2017-12-27 | 2017-12-27 | A kind of localization method based on signboard image, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711447370.XA CN108197619A (en) | 2017-12-27 | 2017-12-27 | A kind of localization method based on signboard image, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108197619A true CN108197619A (en) | 2018-06-22 |
Family
ID=62584697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711447370.XA Pending CN108197619A (en) | 2017-12-27 | 2017-12-27 | A kind of localization method based on signboard image, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108197619A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109345589A (en) * | 2018-09-11 | 2019-02-15 | 百度在线网络技术（北京）有限公司 | Method for detecting position, device, equipment and medium based on automatic driving vehicle |
| CN109389612A (en) * | 2018-08-31 | 2019-02-26 | 南京理工大学 | A kind of municipal rail train pantograph pan edge detection method |
| CN109582880A (en) * | 2018-12-04 | 2019-04-05 | 百度在线网络技术（北京）有限公司 | Interest point information processing method, device, terminal and storage medium |
| CN110309715A (en) * | 2019-05-22 | 2019-10-08 | 北京邮电大学 | Indoor orientation method, the apparatus and system of lamps and lanterns identification based on deep learning |
| CN111078552A (en) * | 2019-12-16 | 2020-04-28 | 腾讯科技（深圳）有限公司 | Method and device for detecting page display abnormity and storage medium |
| CN112070833A (en) * | 2020-08-07 | 2020-12-11 | 北京三快在线科技有限公司 | Signboard map generation method and device and path planning method and device |
| CN109543680B (en) * | 2018-11-19 | 2021-03-12 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device, and medium for determining location of point of interest |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726284A (en) * | 2008-10-29 | 2010-06-09 | 宏碁股份有限公司 | Place identification method, photographing device and place identification system |
| US20140257688A1 (en) * | 2013-03-11 | 2014-09-11 | Qualcomm Incorporated | Methods and apparatus for position estimation |
| CN104977011A (en) * | 2014-04-09 | 2015-10-14 | 广东融讯信息科技有限公司 | Positioning method and positioning device based on street-photographing image in electronic map |
| CN107392214A (en) * | 2017-07-26 | 2017-11-24 | 成都快眼科技有限公司 | A kind of object detection method based on full convolution splitting network |
-
2017
- 2017-12-27 CN CN201711447370.XA patent/CN108197619A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726284A (en) * | 2008-10-29 | 2010-06-09 | 宏碁股份有限公司 | Place identification method, photographing device and place identification system |
| US20140257688A1 (en) * | 2013-03-11 | 2014-09-11 | Qualcomm Incorporated | Methods and apparatus for position estimation |
| CN104977011A (en) * | 2014-04-09 | 2015-10-14 | 广东融讯信息科技有限公司 | Positioning method and positioning device based on street-photographing image in electronic map |
| CN107392214A (en) * | 2017-07-26 | 2017-11-24 | 成都快眼科技有限公司 | A kind of object detection method based on full convolution splitting network |
Non-Patent Citations (2)
| Title |
|---|
| JIFENG DAI ET AL: "Deformable Convolutional Networks", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
| YUANYUAN LI ET AL: "Graphic Logo Detection with Deep Region-based Convolutional Networks", 《IEEE VISUAL COMMUNICATIONS AND IMAGE PROCESSING》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109389612A (en) * | 2018-08-31 | 2019-02-26 | 南京理工大学 | A kind of municipal rail train pantograph pan edge detection method |
| CN109345589A (en) * | 2018-09-11 | 2019-02-15 | 百度在线网络技术（北京）有限公司 | Method for detecting position, device, equipment and medium based on automatic driving vehicle |
| JP2020042015A (en) * | 2018-09-11 | 2020-03-19 | 百度在線網絡技術（北京）有限公司 | Position detection method of autonomous vehicle, position detection apparatus of autonomous vehicle, and computer readable storage medium |
| US11170525B2 (en) | 2018-09-11 | 2021-11-09 | Baidu Online Network Technology (Beijing) Co., Ltd. | Autonomous vehicle based position detection method and apparatus, device and medium |
| CN109543680B (en) * | 2018-11-19 | 2021-03-12 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device, and medium for determining location of point of interest |
| CN109582880A (en) * | 2018-12-04 | 2019-04-05 | 百度在线网络技术（北京）有限公司 | Interest point information processing method, device, terminal and storage medium |
| CN109582880B (en) * | 2018-12-04 | 2021-07-09 | 百度在线网络技术（北京）有限公司 | Interest point information processing method, device, terminal and storage medium |
| CN110309715A (en) * | 2019-05-22 | 2019-10-08 | 北京邮电大学 | Indoor orientation method, the apparatus and system of lamps and lanterns identification based on deep learning |
| CN110309715B (en) * | 2019-05-22 | 2021-05-25 | 北京邮电大学 | Deep learning-based indoor positioning method, device and system for lamp identification |
| CN111078552A (en) * | 2019-12-16 | 2020-04-28 | 腾讯科技（深圳）有限公司 | Method and device for detecting page display abnormity and storage medium |
| CN112070833A (en) * | 2020-08-07 | 2020-12-11 | 北京三快在线科技有限公司 | Signboard map generation method and device and path planning method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108197619A (en) | A kind of localization method based on signboard image, device, equipment and storage medium | |
| CN102884400B (en) | Messaging device, information processing system and program | |
| Batty et al. | Visualizing the city: communicating urban design to planners and decision-makers | |
| KR101876481B1 (en) | View dependent techniques to determine user interest in a feature in a 3d application | |
| CN101763607B (en) | Online exhibition platform system constructed by using panoramic electronic map and construction method thereof | |
| CN102754097B (en) | Method and apparatus for presenting a first-person world view of content | |
| Ferster et al. | A review of earth observation using mobile personal communication devices | |
| JP5334911B2 (en) | 3D map image generation program and 3D map image generation system | |
| CN104517001A (en) | Browser-based method for displaying to-be-constructed construction information | |
| CN103703758A (en) | Mobile augmented reality system | |
| Miles et al. | Alternative representations of 3D-reconstructed heritage data | |
| JP6985777B1 (en) | Educational service provision methods and equipment using satellite images of artificial intelligence infrastructure | |
| CN107656961A (en) | A kind of method for information display and device | |
| US9135338B2 (en) | Systems and methods for efficient feature based image and video analysis | |
| Xu et al. | Building height calculation for an urban area based on street view images and deep learning | |
| Hu et al. | Hybrid three-dimensional representation based on panoramic images and three-dimensional models for a virtual museum: Data collection, model, and visualization | |
| CN111832579A (en) | Map interest point data processing method and device, electronic equipment and readable medium | |
| CN110309247A (en) | The processing method and processing device of step counting data | |
| Bulbul et al. | Populating virtual cities using social media | |
| Camara | Environmental systems: A multidimensional approach | |
| CN115328320B (en) | Hydraulic engineering online learning method and system | |
| KR102219232B1 (en) | Method and system for providing information of artist to a user using mural | |
| Hong et al. | The use of CCTV in the emergency response: A 3D GIS perspective | |
| Chima et al. | Assessment of Nigeriasat-1 satellite data for urban land use/land cover analysis using object-based image analysis in Abuja, Nigeria | |
| KR100462496B1 (en) | Geographical image information service method using internet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180622 |
|
| RJ01 | Rejection of invention patent application after publication |