Выделить слова:


Патент США №	10181080
Автор(ы)	Schultz и др.
Дата выдачи	15 января 2019 г.

Unmanned aircraft structure evaluation system and method

РЕФЕРАТ

Methods and systems are disclosed including a computer storage medium, comprising instructions that when executed by one or more processors included in an Unmanned Aerial Vehicle (UAV), cause the UAV to perform operations, comprising: receiving, by the UAV, a flight plan configured to direct the UAV to fly a flight path having a plurality of waypoints adjacent to and above a structure and to capture sensor data of the structure from a camera on the UAV while the UAV is flying the flight path; adjusting an angle of an optical axis of the camera mounted to a gimbal to a predetermined angle within a range of 25 degrees to 75 degrees relative to a downward direction, and capturing sensor data of at least a portion of a roof of the structure with the optical axis of the camera aligned with at least one predetermined location on the structure.

Авторы:

Stephen L. Schultz (West Henrietta, NY), John Monaco (Penfield, NY)

Патентообладатель:

Имя	Город	Штат	Страна	Тип
Pictometry International Corp.	Rochester	NY	US

Заявитель:

Pictometry International Corp. (Rochester, NY)

ID семейства патентов

53524310

Номер заявки:

16/048,537

Дата регистрации:

30 июля 2018 г.

Prior Publication Data


	Document Identifier	Publication Date
	US 20180357477 A1	Dec 13, 2018

Отсылочные патентные документы США


Application Number	Filing Date	Patent Number	Issue Date
15803129	Nov 3, 2017	10037464
15475978	Mar 31, 2017
14591556	Apr 4, 2017	9612598
61926137	Jan 10, 2014

Класс патентной классификации США:	1/1
Класс совместной патентной классификации:	B64D 47/08 (20130101); G08G 5/0086 (20130101); G06K 9/00637 (20130101); H04N 5/44591 (20130101); G05D 1/0094 (20130101); G01S 19/39 (20130101); B60R 1/00 (20130101); G06T 11/60 (20130101); G08G 5/0039 (20130101); G06K 9/0063 (20130101); G06F 16/51 (20190101); G06F 16/5838 (20190101); G06F 16/5866 (20190101); G08G 5/045 (20130101); G08G 5/0069 (20130101); B64C 39/024 (20130101); B64C 2201/123 (20130101); G06Q 40/08 (20130101); B64C 2201/127 (20130101); B64C 2201/141 (20130101); B64C 2201/146 (20130101); B60R 2300/8093 (20130101)
Класс международной патентной классификации (МПК):	G06K 9/00 (20060101); G05D 1/00 (20060101); G08G 5/00 (20060101); B64C 39/02 (20060101); B64D 47/08 (20060101); H04N 5/445 (20110101); G06Q 40/08 (20120101)
Область поиска:	;701/3

Использованные источники

[Referenced By]

Патентные документы США


2273876	February 1942	Lutz et al.
3153784	October 1964	Petrides et al.
3594556	July 1971	Edwards
3614410	October 1971	Bailey
3621326	November 1971	Hobrough
3661061	May 1972	Tokarz
3716669	February 1973	Watanabe et al.
3725563	April 1973	Woycechowsky
3864513	February 1975	Halajian et al.
3866602	February 1975	Furihata
3877799	April 1975	O'Donnell
4015080	March 1977	Moore-Searson
4044879	August 1977	Stahl
4184711	January 1980	Wakimoto
4240108	December 1980	Levy
4281354	July 1981	Conte
4344683	August 1982	Stemme
4360876	November 1982	Girault et al.
4382678	May 1983	Thompson et al.
4387056	June 1983	Stowe
4396942	August 1983	Gates
4463380	July 1984	Hooks
4489322	December 1984	Zulch et al.
4490742	December 1984	Wurtzinger
4491399	January 1985	Bell
4495500	January 1985	Vickers
4527055	July 1985	Harkless et al.
4543603	September 1985	Laures
4586138	April 1986	Mullenhoff et al.
4635136	January 1987	Ciampa et al.
4653136	March 1987	Denison
4653316	March 1987	Fukuhara
4673988	June 1987	Jansson et al.
4686474	August 1987	Olsen et al.
4688092	August 1987	Kamel et al.
4689748	August 1987	Hofmann
4707698	November 1987	Constant et al.
4758850	July 1988	Archdale et al.
4805033	February 1989	Nishikawa
4807024	February 1989	Mclaurin et al.
4814711	March 1989	Olsen et al.
4814896	March 1989	Heitzman et al.
4843463	June 1989	Michetti
4899296	February 1990	Khattak
4906198	March 1990	Cosimano et al.
4953227	August 1990	Katsuma et al.
4956872	September 1990	Kimura
5034812	July 1991	Rawlings
5086314	February 1992	Aoki et al.
5121222	June 1992	Endoh et al.
5138444	August 1992	Hiramatsu
5155597	October 1992	Lareau et al.
5164825	November 1992	Kobayashi et al.
5166789	November 1992	Myrick
5191174	March 1993	Chang et al.
5200793	April 1993	Ulich et al.
5210586	May 1993	Grage et al.
5231435	July 1993	Blakely
5247356	September 1993	Ciampa
5251037	October 1993	Busenberg
5265173	November 1993	Griffin et al.
5267042	November 1993	Tsuchiya et al.
5270756	December 1993	Busenberg
5296884	March 1994	Honda et al.
5335072	August 1994	Tanaka et al.
5342999	August 1994	Frei et al.
5345086	September 1994	Bertram
5353055	October 1994	Hiramatsu
5369443	November 1994	Woodham
5402170	March 1995	Parulski et al.
5414462	May 1995	Veatch
5467271	November 1995	Abel et al.
5481479	January 1996	Wight et al.
5486948	January 1996	Imai et al.
5506644	April 1996	Suzuki et al.
5508736	April 1996	Cooper
5555018	September 1996	von Braun
5581250	December 1996	Khvilivitzky
5604534	February 1997	Hedges et al.
5617224	April 1997	Ichikawa et al.
5633946	May 1997	Lachinski et al.
5668593	September 1997	Lareau et al.
5677515	October 1997	Selk et al.
5798786	August 1998	Lareau et al.
5835133	November 1998	Moreton et al.
5841574	November 1998	Willey
5844602	December 1998	Lareau et al.
5852753	December 1998	Lo et al.
5894323	April 1999	Kain et al.
5899945	May 1999	Baylocq et al.
5963664	October 1999	Kumar et al.
6037945	March 2000	Loveland
6088055	July 2000	Lareau et al.
6094215	July 2000	Sundahl et al.
6097854	August 2000	Szeliski et al.
6108032	August 2000	Hoagland
6130705	October 2000	Lareau et al.
6157747	December 2000	Szeliski et al.
6167300	December 2000	Cherepenin et al.
6222583	April 2001	Matsumura et al.
6236886	May 2001	Cherepenin et al.
6256057	July 2001	Mathews et al.
6373522	April 2002	Mathews et al.
6421610	July 2002	Carroll et al.
6434280	August 2002	Peleg et al.
6597818	July 2003	Kumar et al.
6639596	October 2003	Shum et al.
6711475	March 2004	Murphy
6731329	May 2004	Feist et al.
6747686	June 2004	Bennett
6810383	October 2004	Loveland
6816819	November 2004	Loveland
6826539	November 2004	Loveland
6829584	December 2004	Loveland
6834128	December 2004	Altunbasak et al.
6876763	April 2005	Sorek et al.
7009638	March 2006	Gruber et al.
7018050	March 2006	Ulichney et al.
7046401	May 2006	Dufaux et al.
7061650	June 2006	Walmsley et al.
7065260	June 2006	Zhang et al.
7123382	October 2006	Walmsley et al.
7127348	October 2006	Smitherman et al.
7133551	November 2006	Chen
7142984	November 2006	Rahmes et al.
7184072	February 2007	Loewen et al.
7233691	June 2007	Setterholm
7262790	August 2007	Bakewell
7348895	March 2008	Lagassey
7509241	March 2009	Guo
7728833	June 2010	Verma
7832267	November 2010	Woro
7844499	November 2010	Yahiro
8078396	December 2011	Meadow
8209152	June 2012	Pershing
8346578	January 2013	Hopkins, III et al.
8422825	April 2013	Neophytou et al.
8473125	June 2013	Rischmuller et al.
8705843	April 2014	Lieckfeldt
8818572	August 2014	Tofte et al.
8874283	October 2014	Cavote
8931144	January 2015	Freeman et al.
8991758	March 2015	Earon
9162753	October 2015	Panto et al.
9250630	February 2016	Downey et al.
9256225	February 2016	Downey et al.
9256994	February 2016	Downey et al.
9273981	March 2016	Downey et al.
9310221	April 2016	Downey et al.
9311760	April 2016	Downey et al.
9330504	May 2016	Ege
9340283	May 2016	Downey et al.
9403593	August 2016	Downey et al.
9406237	August 2016	Downey et al.
9437033	September 2016	Sun et al.
9437044	September 2016	Ege et al.
9501760	November 2016	Huang et al.
9513635	December 2016	Bethke et al.
9588516	March 2017	Gurel et al.
9592912	March 2017	Michini et al.
9607522	March 2017	Downey et al.
9609288	March 2017	Richman et al.
9613538	April 2017	Poole et al.
9615014	April 2017	Kim
9618940	April 2017	Michini et al.
9658619	May 2017	Bethke et al.
9679227	June 2017	Taylor et al.
9851716	December 2017	Kugelmass
2002/0041328	April 2002	LeCompte et al.
2002/0041717	April 2002	Murata et al.
2002/0114536	August 2002	Xiong et al.
2003/0014224	January 2003	Guo et al.
2003/0043824	March 2003	Remboski et al.
2003/0088362	May 2003	Melero et al.
2003/0164962	September 2003	Nims et al.
2003/0214585	November 2003	Bakewell
2004/0105090	June 2004	Schultz et al.
2004/0167709	August 2004	Smitherman et al.
2005/0073241	April 2005	Yamauchi et al.
2005/0088251	April 2005	Matsumoto
2005/0169521	August 2005	Hel-Or
2006/0028550	February 2006	Palmer et al.
2006/0092043	May 2006	Lagassey
2006/0238383	October 2006	Kimchi et al.
2006/0250515	November 2006	Koseki et al.
2007/0024612	February 2007	Balfour
2007/0046448	March 2007	Smitherman
2007/0237420	October 2007	Steedly et al.
2007/0244608	October 2007	Rath et al.
2008/0120031	May 2008	Rosenfeld et al.
2008/0123994	May 2008	Schultz et al.
2008/0158256	July 2008	Russell et al.
2009/0177458	July 2009	Hochart et al.
2009/0208095	August 2009	Zebedin
2009/0259350	October 2009	Morris et al.
2009/0265193	October 2009	Collins et al.
2009/0304227	December 2009	Kennedy et al.
2010/0017114	January 2010	Tehan et al.
2010/0042269	February 2010	Kokkeby et al.
2010/0100269	April 2010	Ekhaguere
2010/0121574	May 2010	Ariyur et al.
2010/0215212	August 2010	Flakes
2010/0250022	September 2010	Hines et al.
2010/0286859	November 2010	Feigh et al.
2010/0296693	November 2010	Thornberry et al.
2011/0033110	February 2011	Shimamura et al.
2012/0143482	June 2012	Goossen et al.
2012/0232722	September 2012	Fisher et al.
2013/0162822	June 2013	Lee et al.
2013/0216089	August 2013	Chen et al.
2013/0246204	September 2013	Thornberry et al.
2013/0317667	November 2013	Kruglick
2014/0018979	January 2014	Goossen et al.
2014/0168420	June 2014	Naderhirn et al.
2014/0316614	October 2014	Newman
2014/0316616	October 2014	Kugelmass
2015/0022656	January 2015	Carr et al.
2015/0225081	August 2015	Stabler et al.
2015/0254738	September 2015	Wright et al.
2015/0257424	September 2015	Catani et al.
2016/0148363	May 2016	Phan et al.
2016/0187882	June 2016	Downey et al.
2016/0257424	September 2016	Stabler et al.
2016/0285774	September 2016	Downey et al.
2016/0301859	October 2016	Tebay
2016/0307447	October 2016	Johnson et al.
2017/0154535	June 2017	Downey et al.
2017/0192418	July 2017	Bethke et al.
2017/0192424	July 2017	Poole et al.
2017/0193297	July 2017	Michini et al.
2017/0193829	July 2017	Bauer et al.
2017/0199647	July 2017	Richman et al.

Зарубежные патентные документы


331204	Jul 2006	AT
0316110	Sep 2005	BR
2402234	Sep 2000	CA
2505566	May 2004	CA
1735897	Feb 2006	CN
60017384	Mar 2006	DE
60306301	Nov 2006	DE
1418402	Oct 2006	DK
1010966	Feb 1999	EP
1180967	Feb 2002	EP
1418402	May 2004	EP
1696204	Aug 2006	EP
2244150	Oct 2010	EP
2266704	Mar 2007	ES
2000/175183	Jun 2000	JP
2000/341672	Dec 2000	JP
2003/317089	Nov 2003	JP
10-2012-0060340	Jun 2012	KR
PA05004987	Feb 2006	MX
1999/18732	Apr 1999	WO
2000/053090	Sep 2000	WO
2004/044692	May 2004	WO
2005/088251	Sep 2005	WO
2008/028040	Mar 2008	WO
WO 2016/131005	Aug 2016	WO
WO 2017/116860	Jul 2017	WO

Другие источники

USPTO, Office Action regarding U.S. Appl. No. 16/049,253, dated Sep. 28, 2018. cited by applicant .
Ackermann, Prospects of Kinematic GPS Aerial Triangulation, ITC Journal, 1992. cited by applicant .
Ciampa, John A., "Pictometry Digital Video Mapping", SPIE, vol. 2598, pp. 140-148, 1995. cited by applicant .
Ciampa, J. A., Oversee, Presented at Reconstruction After Urban earthquakes, Buffalo, NY, 1989. cited by applicant .
Dunford et al., Remote Sensing for Rural Development Planning in Africa, The Journal for the International Institute for Aerial Survey and Earth Sciences, 2:99-108, 1983. cited by applicant .
Gagnon, P.A., Agnard, J. P., Nolette, C., & Boulianne, M., "A Micro-Computer based General Photogrammetric System", Photogrammetric Engineering and Remote Sensing, vol. 56, No. 5., pp. 623-625, 1990. cited by applicant .
Konecny, G., "Issues of Digital Mapping", Leibniz University Hannover, Germany, GIS Ostrava 2008, Ostrava 27.--Jan. 30, 2008, pp. 1-8. cited by applicant .
Konecny, G., "Analytical Aerial Triangulation with Convergent Photography", Department of Surveying Engineering, University of New Brunswick, pp. 37-57, 1966. cited by applicant .
Konecny, G., "Interior Orientation and Convergent Photography", Photogrammetric Engineering, pp. 625-634, 1965. cited by applicant .
Graham, Lee A., "Airborne Video for Near-Real-Time Vegetation Mapping", Journal of Forestry, 8:28-32, 1993. cited by applicant .
Graham, Horita TRG-50 SMPTE Time-Code Reader, Generator, Window Inserter, 1990. cited by applicant .
Hess, L.L, et al., "Geocoded Digital Videography for Validation of Land Cover Mapping in the Amazon Basin", International Journal of Remote Sensing, vol. 23, No. 7, pp. 1527-1555, 2002. cited by applicant .
Hinthorne, J., et al., "Image Processing in The Grass GIS", Geoscience and Remote Sensing Symposium, 4:2227-2229, 1991. cited by applicant .
Imhof, Ralph K., "Mapping from Oblique Photographs", Manual of Photogrammetry, Chapter 18, 1966. cited by applicant .
Jensen, John R., Introductory Digital Image Processing: A Remote Sensing Perspective, Prentice-Hall, 1986; 399 pages. cited by applicant .
Lapine, Lewis A., "Practical Photogrammetric Control by Kinematic GPS", GPS World, 1(3):44-49, 1990. cited by applicant .
Lapine, Lewis A., Airborne Kinematic GPS Positioning for Photogrammetry--The Determination of the Camera Exposure Station, Silver Spring, MD, 11 pages, at least as early as 2000. cited by applicant .
Linden et al., Airborne Video Automated Processing, US Forest Service Internal report, Fort Collins, CO, 1993. cited by applicant .
Myhre, Dick, "Airborne Video System Users Guide", USDA Forest Service, Forest Pest Management Applications Group, published by Management Assistance Corporation of America, 6 pages, 1992. cited by applicant .
Myhre et al., "An Airborne Video System Developed Within Forest Pest Management--Status and Activities", 10 pages, 1992. cited by applicant .
Myhre et al., "Airborne Videography--A Potential Tool for Resource Managers"--Proceedings: Resource Technology 90, 2nd International Symposium on Advanced Technology in Natural Resource Management, 5 pages, 1990. cited by applicant .
Myhre et al., Aerial Photography for Forest Pest Management, Proceedings of Second Forest Service Remote Sensing Applications Conference, Slidell, Louisiana, 153-162, 1988. cited by applicant .
Myhre et al., "Airborne Video Technology", Forest Pest Management/Methods Application Group, Fort Collins, CO, pp. 1-6, at least as early as Jul. 30, 2006. cited by applicant .
Norton-Griffiths et al., 1982. "Sample surveys from light aircraft combining visual observations and very large scale color photography". University of Arizona Remote Sensing Newsletter 82-2:1-4. cited by applicant .
Norton-Griffiths et al., "Aerial Point Sampling for Land Use Surveys", Journal of Biogeography, 15:149-156, 1988. cited by applicant .
Novak, Rectification of Digital Imagery, Photogrammetric Engineering and Remote Sensing, 339-344, 1992. cited by applicant .
Slaymaker, Dana M., "Point Sampling Surveys with GPS-logged Aerial Videography", Gap Bulletin No. 5, University of Idaho, http://www.gap.uidaho.edu/Bulletins/5/PSSwGPS.html, 1996. cited by applicant .
Slaymaker, et al., "Madagascar Protected Areas Mapped with GPS-logged Aerial Video and 35mm Air Photos", Earth Observation magazine, vol. 9, No. 1, http://www.eomonline.com/Common/Archives/2000jan/00jan_tableofcont- ents.html, pp. 1-4, 2000. cited by applicant .
Slaymaker, et al., "Cost-effective Determination of Biomass from Aerial Images", Lecture Notes in Computer Science, 1737:67-76, http://portal.acm.org/citation.cfm?id=648004.743267&coll=GUIDE&dl=, 1999. cited by applicant .
Slaymaker, et al., "A System for Real-time Generation of Geo-referenced Terrain Models", 4232A-08, SPIE Enabling Technologies for Law Enforcement Boston, MA, ftp://vis-ftp.cs.umass.edu/Papers/schultz/spie2000.pdf, 2000. cited by applicant .
Slaymaker, et al.,"Integrating Small Format Aerial Photography, Videography, and a Laser Profiler for Environmental Monitoring", in ISPRS WG III/1 Workshop on Integrated Sensor Calibration and Orientation, Portland, Maine, 1999. cited by applicant .
Slaymaker, et al., "Calculating Forest Biomass With Small Format Aerial Photography, Videography and a Profiling Laser", In Proceedings of the 17th Biennial Workshop on Color Photography and Videography in Resource Assessment, Reno, NV, 1999. cited by applicant .
Slaymaker et al., Mapping Deciduous Forests in Southern New England using Aerial Videography and Hyperclustered Multi-Temporal Landsat TM Imagery, Department of Forestry and Wildlife Management, University of Massachusetts, 1996. cited by applicant .
Star et al., "Geographic Information Systems an Introduction", Prentice-Hall, 1990. cited by applicant .
Tomasi et al., "Shape and Motion from Image Streams: a Factorization Method"--Full Report on the Orthographic Case, pp. 9795-9802, 1992. cited by applicant .
Warren, Fire Mapping with the Fire Mousetrap, Aviation and Fire Management, Advanced Electronics System Development Group, USDA Forest Service, 1986. cited by applicant .
Welch, R., "Desktop Mapping with Personal Computers", Photogrammetric Engineering and Remote Sensing, 1651-1662, 1989. cited by applicant .
Westervelt, James, "Introduction to Grass 4", pp. 1-25, 1991. cited by applicant .
"RGB Spectrum Videographics Report, vol. 4, No. 1, McDonnell Douglas Integrates RGB Spectrum Systems in Helicopter Simulators", pp. 1-6, 1995. cited by applicant .
RGB "Computer Wall", RGB Spectrum, 4 pages, 1995. cited by applicant .
"The First Scan Converter with Digital Video Output", Introducing . . . The RGB/Videolink 1700D-1, RGB Spectrum, 2 pages, 1995. cited by applicant .
ERDAS Field Guide, Version 7.4, A Manual for a commercial image processing system, 1990. cited by applicant .
"Image Measurement and Aerial Photography", Magazine for all branches of Photogrammetry and its fringe areas, Organ of the German Photogrammetry Association, Berlin-Wilmersdorf, No. 1, 1958. cited by applicant .
"Airvideo Analysis", MicroImages, Inc., Lincoln, NE, 1 page, Dec. 1992. cited by applicant .
Zhu, Zhigang, Hanson, Allen R., "Mosaic-Based 3D Scene Representation and Rendering", Image Processing, 2005, ICIP 2005, IEEE International Conference on 1(2005). cited by applicant .
Mostafa, et al., "Direct Positioning and Orientation Systems How do they Work? What is the Attainable Accuracy?", Proceeding, American Society of Photogrammetry and Remote Sensing Annual Meeting, St. Louis, MO, Apr. 24-27, 2001. cited by applicant .
"POS AV" georeferenced by APPLANIX aided inertial technology, http://www.applanix.com/products/posav_index.php. cited by applicant .
Mostafa, et al., "Ground Accuracy from Directly Georeferenced Imagery", Published in GIM International vol. 14 N. 12 Dec. 2000. cited by applicant .
Mostafa, et al., "Airborne Direct Georeferencing of Frame Imagery: An Error Budget", The 3.sup.rd International Symposium on Mobile Mapping Technology, Cairo, Egypt, Jan. 3-5, 2001. cited by applicant .
Mostafa, M.R. and Hutton, J., "Airborne Kinematic Positioning and Attitude Determination Without Base Stations", Proceedings, International Symposium on Kinematic Systems in Geodesy, Geomatics, and Navigation (KIS 2001) Banff, Alberta, Canada, Jun. 4-8, 2001. cited by applicant .
Mostafa, et al., "Airborne DGPS Without Dedicated Base Stations for Mapping Applications", Proceedings of ION-GPS 2001, Salt Lake City, Utah, USA, Sep. 11-14. cited by applicant .
Mostafa, "ISAT Direct Exterior Orientation QA/QC Strategy Using POS Data", Proceedings of OEEPE Workshop: Integrated Sensor Orientation, Hanover, Germany, Sep. 17-18, 2001. cited by applicant .
Mostafa, "Camera/IMU Boresight Calibration: New Advances and Performance Analysis", Proceedings of the ASPRS Annual Meeting, Washington, D.C., Apr. 21-26, 2002. cited by applicant .
Hiatt, "Sensor Integration Aids Mapping at Ground Zero", Photogrammetric Engineering and Remote Sensing, Sep. 2002, p. 877-878. cited by applicant .
Mostafa, "Precision Aircraft GPS Positioning Using CORS", Photogrammetric Engineering and Remote Sensing, Nov. 2002, p. 1125-1126. cited by applicant .
Mostafa, et al., System Performance Analysis of INS/DGPS Integrated System for Mobile Mapping System (MMS), Department of Geomatics Engineering, University of Calgary, Commission VI, WG VI/4, Mar. 2004. cited by applicant .
Artes F., & Hutton, J., "GPS and Inertial Navigation Delivering", Sep. 2005, GEOconnexion International Magazine, p. 52-53, Sep. 2005. cited by applicant .
"POS AV" APPLANIX, Product Outline, airborne@applanix.com, 3 pages, Mar. 28, 2007. cited by applicant .
POSTrack, "Factsheet", APPLANIX, Ontario, Canada, www.applanix.com, Mar. 2007. cited by applicant .
POS AV "Digital Frame Camera Applications", 3001 Inc., Brochure, 2007. cited by applicant .
POS AV "Digital Scanner Applications", Earthdata Brochure, Mar. 2007. cited by applicant .
POS AV "Film Camera Applications" AeroMap Brochure, Mar. 2007. cited by applicant .
POS AV "LIDAR Applications" MD Atlantic Brochure, Mar. 2007. cited by applicant .
POS AV "OEM System Specifications", 2005. cited by applicant .
POS AV "Synthetic Aperture Radar Applications", Overview, Orbisat Brochure, Mar. 2007. cited by applicant .
"POSTrack V5 Specifications" 2005. cited by applicant .
"Remote Sensing for Resource Inventory Planning and Monitoring", Proceeding of the Second Forest Service Remote Sensing Applications Conference--Slidell, Louisiana and NSTL, Mississippi, Apr. 11-15, 1988. cited by applicant .
"Protecting Natural Resources with Remote Sensing", Proceeding of the Third Forest Service Remote Sensing Applications Conference--Apr. 9-13, 1990. cited by applicant .
Heipke, et al, "Test Goals and Test Set Up for the OEEPE Test--Integrated Sensor Orientation", 1999. cited by applicant .
Kumar, et al., "Registration of Video to Georeferenced Imagery", Sarnoff Corporation, CN5300, Princeton, NJ, 1998. cited by applicant .
McConnel, Proceedings Aerial Pest Detection and Monitoring Workshop--1994.pdf, USDA Forest Service Forest Pest Management, Northern Region, Intermountain region, Forest Insects and Diseases, Pacific Northwest Region. cited by applicant .
"Standards for Digital Orthophotos", National Mapping Program Technical Instructions, US Department of the Interior, Dec. 1996. cited by applicant .
Tao, "Mobile Mapping Technology for Road Network Data Acquisition", Journal of Geospatial Engineering, vol. 2, No. 2, pp. 1-13, 2000. cited by applicant .
"Mobile Mapping Systems Lesson 4", Lesson 4 SURE 382 Geographic Information Systems II, pp. 1-29, Jul. 2, 2006. cited by applicant .
Konecny, G., "Mechanische Radialtriangulation mit Konvergentaufnahmen", Bildmessung and Luftbildwesen, 1958, Nr. 1. cited by applicant .
Myhre, "ASPRS/ACSM/RT 92" Technical papers, Washington, D.C., vol. 5 Resource Technology 92, Aug. 3-8, 1992. cited by applicant .
Rattigan, "Towns get new view from above," The Boston Globe, Sep. 5, 2002. cited by applicant .
Mostafa, et al., "Digital image georeferencing from a multiple camera system by GPS/INS," ISP RS Journal of Photogrammetry & Remote Sensing, 56(I): I-12, Jun. 2001. cited by applicant .
Dillow, "Grin, or bare it, for aerial shot," Orange County Register (California), Feb. 25, 200I. cited by applicant .
Anonymous, "Live automatic coordinates for aerial images," Advanced Imaging, 12(6):51, Jun. 1997. cited by applicant .
Anonymous, "Pictometry and US Geological Survey announce--Cooperative Research and Development Agreement," Press Release published Oct. 20, 1999. cited by applicant .
Miller, "Digital software gives small Arlington the Big Picture," Government Computer NewsState & Local, 7(12), Dec. 2001. cited by applicant .
Garrett, "Pictometry: Aerial photography on steroids," Law Enforcement Technology 29(7):114-116, Jul. 2002. cited by applicant .
Weaver, "County gets an eyeful," The Post-Standard (Syracuse, NY), May 18, 2002. cited by applicant .
Reed, "Firm gets latitude to map O.C. in 3D," Orange County Register (California), Sep. 27, 2000. cited by applicant .
Reyes, "Orange County freezes ambitious aerial photography project," Los Angeles Times, Oct. 16, 2000. cited by applicant .
Aerowest Pricelist of Geodata as of Oct. 21, 2005 and translations to English 3 pages. cited by applicant .
www.archive.org Web site showing archive of German AeroDach Web Site http://www.aerodach.de from Jun. 13, 2004 (retrieved Sep. 20, 2012) and translations to English 4 pages. cited by applicant .
AeroDach.RTM.Online Roof Evaluation Standard Delivery Format and 3D Data File: Document Version 01.00.2002 with publication in 2002, 13 pages. cited by applicant .
Noronha et al., "Detection and Modeling of Building from Multiple Aerial Images, " Institute for Robotics and Intelligent Systems, University of Southern California, Nov. 27, 2001, 32 pages. cited by applicant .
Applicad Reports dated Nov. 25, 1999-Mar. 9, 2005, 50 pages. cited by applicant .
Applicad Online Product Bulletin archive from Jan. 7, 2003, 4 pages. cited by applicant .
Applicad Sorcerer Guide, Version 3, Sep. 8, 1999, 142 pages. cited by applicant .
Xactimate Claims Estimating Software archive from Feb. 12, 2010, 8 pages. cited by applicant .
Bignone et al, Automatic Extraction of Generic House Roofs from High Resolution Aerial Imagery, Communication Technology Laboratory, Swiss Federal Institute of Technology ETH, CH-8092 Zurich, Switzerland, 12 pages, 1996. cited by applicant .
Geospan 2007 Job proposal. cited by applicant .
Greening et al., Commercial Applications of GPS-Assisted Photogrammetry, Presented at GIS/LIS Annual Conference and Exposition, Phoenix, AZ, Oct. 1994. cited by applicant .
Mission Planner Overview [online webpage]; Retrieved from http://ardupilot.org/planner/docs/mission-planner-overview.html; Retrieved Dec. 15, 2016. cited by applicant .
Planning a Mission with Waypoints and Events [online webpage]; Retrieved from http://ardupilot.org/planner/docs/common-planning-a-mission-with-way- points-and-events.html; Retrieved Dec. 15, 2016. cited by applicant .
Kespry Aerial Data System [online webpage]; Retrieved from https://web.archive.org/web/20141118185641/http://kespry.com/; Retrieved Dec. 15, 2016. cited by applicant .
International Search Report and Written Opinion of PCT/US2015/010496, Korean Intellectual Property Office, dated Apr. 20, 2015. cited by applicant .
European Patent Office, Extended European Search Report regarding application EP15734836.8, dated Jun. 7, 2017. cited by applicant .
Chesebro, Jonathan; "Unmanned Aircraft Systems (UAS)" [online website article] Retrieved from http://www.trade.gov/static/aero_rpt_flight_plan_2011_uas.pdf, [Retrieved May 19, 2015]. cited by applicant .
Brown et al.; "Unmanned Drones--Technical Elegance vs. Political Consequences;" TheSciTechLawyer; vol. 7, No. 1, Summer 2010, American Bar Association. cited by applicant .
Cloar et al.; "Unmanned Aerial Systems: Mobility on the Edge;" TheSciTechLawyer; vol. 9, Nos. 3 & 4, Winter Spring 2013, American Bar Association; Retrieved from http://www.americanbar.org/publications/scitech_lawyer/2013/winter_spring- _2013/unmanned_aerial_systems_mobility_on_the_edge.html [Retrieved Jun. 8, 2015]. cited by applicant .
Whitlock, Craig; "Close Encounters on Rise as Small Drones Gain in Popularity;" Jun. 23, 2014; The Washington Post; Retrieved from http://www.washingtonpost.com/sf/investigative/2014/06/23/close-encounter- s-with-small-drones-on-rise/ [Retrieved May 18, 2015]. cited by applicant .
Whitlock, Craig; "Crashes Mount as Military Flies More Drones in U.S.;" Jun. 22, 2014; The Washington Post; Retrieved from http://www.washingtonpost.com/sf/investigative/2014/06/22/crashes-mount-a- s-military-flies-more-drones-in-u-s/ [Retrieved May 18, 2015]. cited by applicant .
Drone Technology; "RPAS MCFLY-IR;" [Website] Retrieved from http://www.dronetechnology.eu/rpasmcflyir/ [Retrieved Jun. 15, 2015]. cited by applicant .
Johnson, Ted; "Filmmakers Get Permission to Use Drones in the U.S."; Variety; Sep. 25, 2014. cited by applicant .
Curran, Oisin; "How Drone Cinematography Works"; [Online Article] Retrieved from http://entertainment.howstuffworks.com/dronecinematography5.htm/printable [Retrieved May 5, 2015]. cited by applicant .
Anonymous; "Know Before You Fly" Brochure; Retrieved from http://knowbeforeyoufly.org/wp-content/uploads/2015/12/KBYF_Brochure_WEB.- pdf [Retrieved May 5, 2015]. cited by applicant .
Anonymous; "Know Before You Fly" Website; Retrieved from http://knowbeforeyoufly.org/about-us/ [Retrieved May 5, 2015]. cited by applicant .
Anonymous; LATAS Website; Retrieved from http://flylatas.com/ [Retrieved Jun. 2, 2015]. cited by applicant .
McKeever et al.; "Report of the Governor's Oklahoma Unmanned Aerial Systems Council"; Office of the Governor--State of Oklahoma; Spring 2012. cited by applicant .
Calvo, Kike "So You Want to Shoot Aerial Photography Using Drones?"; Mar. 5, 2014; The National Geographic Society; Retrieved from http://voices.nationalgeographic.com/2014/03/05/soyouwanttoshootaerialpho- tographyusingdrones/ [Retrieved May 5, 2015]. cited by applicant .
Whitlock, Craig; "When Drones Fall from the Sky;" Jun. 20, 2014; The Washington Post; Retrieved from http://www.washingtonpost.com/sf/investigative/ 2014/06/20/when-drones-fall-from-the-sky/ [Retrieved May 18, 2015]. cited by applicant .
King, Rachael; "Yamaha Waits for FAA Approval on Agricultural Drone"; CIO Journal; Oct. 16, 2014; Retrieved from http://blogs.wsj.com/cio/2014/10/16/yamahawaitsforfaaapprovalonagricultur- aldrone/ [Retrieved May 19, 2015]. cited by applicant .
Hanlon, Mike; "Yamaha's RMAX--the worlds most advanced non-military UAV"; Jun. 4, 2004 updated Nov. 19, 2004; Gizmag; Retrieved from http://www.gizmag.com/go/2440/ [Retrieved May 19, 2015]. cited by applicant .
Fathi, Habib; "Videogrammetric Roof Surveying Using a Hybrid Structure from Motion Approach"; Georgia Institute of Technology; Dec. 1, 2013; Retrieved from https://smartech.gatech.edu/handle/1853/52972 [Retrieved Dec. 1, 2017]. cited by applicant.

Главный эксперт: Black; Thomas G
Assistant Examiner: Huynh; Luat T
Уполномоченный, доверенный или фирма: Dunlap Codding, P.C.

Текст решения-прецедента

CROSS REFERENCE TO RELATED APPLICATION/INCORPORATION BY REFERENCE

The present patent application is a continuation of U.S. patent application Ser. No. 15/803,129, filed Nov. 3, 2017, which is a continuation of U.S. patent application Ser. No. 15/475,978, filed Mar. 31, 2017, which claims priority to U.S. patent application Ser. No. 14/591,556, filed Jan. 7, 2015, which issued as U.S. Pat. No. 9,612,598, which claims priority to the provisional patent application identified by U.S. Ser. No. 61/926,137, filed on Jan. 10, 2014, the entire contents of each of which are hereby expressly incorporated by reference herein.

ФОРМУЛА ИЗОБРЕТЕНИЯ

What is claimed is:

1. A non-transitory computer storage medium comprising instructions that when executed by a processor included in an Unmanned Aerial Vehicle (UAV) having a computer controlled gimbal mount connected to a camera, cause the UAV to perform operations, comprising: directing the UAV to fly a target path having a plurality of flight capture points laterally and/or vertically offset from geographic positions of an outline of the structure of interest, a first group of flight capture points adjacent to a first side of the outline, and a second group of flight capture points adjacent to a second side of the outline; capturing images of the structure of interest from the camera on the UAV while the UAV is flying the target path at flight capture points spaced along the target path in such a manner as to ensure full coverage of the structure of interest in the images captured by the camera; and adjusting a direction of the camera relative to the structure of interest in three dimensions as the UAV is flying the target path to point the camera at a target capture point on the structure of interest at each flight capture point.

2. The non-transitory computer storage medium of claim 1, wherein adjusting a direction of the camera relative to the structure of interest is defined further as adjusting a yaw of the UAV to assist in pointing the camera at the target capture point.

3. The non-transitory computer storage medium of claim 1, wherein adjusting the direction of the camera relative to the structure of interest is defined further as adjusting the direction of the camera relative to the structure of interest to a predetermined angle within a range of 25-75 degrees relative to a downward direction.

4. The non-transitory computer storage medium of claim 1, wherein a first one of the plurality of flight capture points direct the UAV to capture sensor data of the structure of interest from a first angle relative to the structure of interest and a second one of the plurality of flight capture points directs the UAV to capture sensor data of the structure of interest from a second angle relative to the structure.

5. The non-transitory computer storage medium of claim 1, wherein the flight capture points are spaced along the target path in such a manner as to maintain at least a minimum overlap between adjacent images to ensure full coverage of the structure of interest in the images captured by the camera.

6. The non-transitory computer storage medium of claim 1, wherein the structure of interest is a roof of a building.

7. A computerized system, comprising: a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing software to cause the one or more processors to: determining a first location of a structure to be inspected, the first location having first geographic coordinates; displaying, via the display unit, a first graphical representation of the structure of interest, the first graphical representation comprising a first image describing an aerial view of the structure of interest; receiving an alteration of the first graphical representation to indicate a change of the first geographic coordinates to second geographic coordinates; subsequent to receiving the alteration of the first graphical representation, generate unmanned aircraft information with information pertaining to the structure of interest identified by the second geographic coordinates for a UAV having a camera mounted to a computer controlled gimbal mount to capture images of the structure of interest, the unmanned aircraft information including flight path information, camera control information, and gimbal control information, the flight path information configured to direct an unmanned aircraft to fly a flight path above the structure of interest, the camera control information configured to control the camera to capture aerial images from the camera on the unmanned aircraft while the unmanned aircraft is flying the flight path, the gimbal control information configured to control the computer controlled gimbal mount to align the camera with the structure of interest during capture of the aerial images; receive the aerial images of the structure of interest from the unmanned aircraft; and generate a structure report based at least in part on the aerial images of the structure of interest.

8. The computerized system of claim 7, wherein the at least one predetermined location on the structure of interest is a roof surface and the optical axis of the camera is aligned orthogonal to the roof surface.

9. The computerized system of claim 7, wherein the camera control information includes a plurality of flight capture points adjacent to the structure to cause the camera of the unmanned aerial vehicle to capture the aerial images at the plurality of flight capture points.

10. The computerized system of claim 7, wherein the information pertaining to the structure of interest includes information pertaining to a footprint of the structure of interest, the footprint being a two-dimensional boundary of the structure of interest.

11. The computerized system of claim 7, wherein the information pertaining to the structure of interest includes information pertaining to a footprint of the structure of interest, the footprint being an outline of the structure of interest.

12. The computerized system of claim 7, wherein the information pertaining to the structure of interest includes information pertaining to a height of the structure of interest, the information pertaining to the height of the structure being used by the software to aid in determining altitude for the unmanned aircraft information.

13. The computerized system of claim 12, wherein the information pertaining to the height of the structure of interest is received from a user via the input unit.

14. The computerized system of claim 7, wherein the information pertaining to the structure of interest include geographic locations of obstructions in potential flight paths of the UAV, and wherein the flight path information is generated to direct the unmanned aircraft to fly the flight path around the structure of interest, and avoid the obstructions.

15. The computerized system of claim 14, wherein the one or more processors execute software to cause the one or more processors to detect the geographic location of obstructions in potential flight paths of the UAV through the use of imagery.

16. The computerized system of claim 7, wherein the second geographic coordinates are indicative of a validated location of the structure of interest.

17. The computerized system of claim 7, wherein the structure of interest is a roof of a building.

18. A computerized system, comprising: a computer system having an input unit, a display unit, one or more processors and one or more non-transitory computer readable medium, the one or more processors executing software to cause the one or more processors to: display, via the display unit, a first flight path on a graphical representation of an external form of a structure of interest, in which an unmanned aircraft is to navigate about the structure of interest; receive data indicative of an outline of the structure of interest, via the input unit, and using the data to generate unmanned aircraft information having a second flight path and gimbal control information, the second flight path configured to direct the unmanned aircraft through autonomous flight above the structure of interest, the gimbal control information configured to align a camera supported by the unmanned aircraft with the structure of interest during capture of the aerial images; display, via the display unit, the second flight path, on the graphical representation of the external form of the structure of interest; receive aerial images of the structure of interest from the unmanned aircraft, the aerial images obtained by the camera when aligned with the structure of interest during capture of the aerial images while the unmanned aircraft is being directed through the second flight path during autonomous flight; and generate a structure report based at least in part on the aerial images of the structure of interest obtained by the camera carried by the unmanned aircraft while the aircraft is being directed through the second flight path.

19. The computerized system of claim 18, wherein the second flight path includes a plurality of spatially disposed flight capture points, the spacing of the flight capture points maintaining at least a minimum overlap between adjacent aerial images to ensure full coverage of the structure of interest in the aerial images captured by the camera.

20. The computerized system of claim 18, wherein the structure of interest has an outline, and wherein the second flight path extends outside the outline of the structure of interest.

21. The computerized system of claim 19, wherein the second flight path extends entirely beyond the outline of the structure of interest.

22. The computerized system of claim 18, wherein the first flight path is determined automatically by analyzing and using geo-referenced images.

23. The computerized system of claim 18, wherein the structure of interest is a roof of a building.

24. An Unmanned Aerial Vehicle (UAV) having a computer controlled gimbal mount connected to a camera, a processor, and a non-transitory computer storage medium comprising instructions that when executed by the processor cause the UAV to perform operations, comprising: directing the UAV to fly a flight path having a plurality of flight capture points laterally and/or vertically offset from geographic positions of an outline of the structure of interest, a first group of flight capture points adjacent to a first side of the outline, and a second group of flight capture points adjacent to a second side of the outline; detecting an obstruction in the flight path, steering the UAV to avoid the obstruction, and adjusting a down-look angle of the camera with the computer controlled gimbal mount as the UAV is avoiding the obstruction; and capturing first images of the structure of interest with the camera on the UAV at the flight capture points while the UAV is flying the flight path, and second images while the UAV is avoiding the obstruction, the first images being overlapping, and the second images being overlapping.

25. The UAV of claim 24, wherein the flight capture points are spaced along the flight path in such a manner as to ensure full coverage of the structure of interest.

26. The UAV of claim 24, wherein an angle of the camera relative to the structure of interest is adjusted to a first angle using the computer controlled gimbal mount while capturing the first images and a second angle relative to the structure of interest while capturing the second images.

27. The UAV of claim 24, wherein the UAV comprises a collision detection and avoidance system to detect the obstruction in the flight path, and to steer the UAV to avoid the obstruction.

28. The UAV of claim 27, wherein the collision detection and avoidance system is imaging based.

29. The UAV of claim 27, wherein the collision detection and avoidance system is active sensor based.

30. The UAV of claim 24, wherein the instructions that when executed by the processor including camera control instructions configured to capture images on a schedule.

ОПИСАНИЕ

УРОВЕНЬ ТЕХНИКИ

Unmanned aerial vehicles (UAVs), commonly known as drones, are aircraft without a human pilot on board. Flight may be controlled by computers or by remote control of a pilot located on the ground.

Within the insurance industry, use of UAVs may aid in obtaining evaluation estimates for structures, such as roofs, that may be difficult to access. For example, a camera may be placed on the UAV so that the roof of a structure may be viewed without having to physically climb onto the roof.

The flight plan of the UAV may be based on evaluation of the geographic area around the structure, and is generally individualized for each structure. Currently within the industry, flight plans and locations of capture images are manually selected by a user.

КРАТКОЕ ОПИСАНИЕ НЕСКОЛЬКИХ ВИДОВ РИСУНКОВ

Like reference numerals in the figures represent and refer to the same or similar element or function. Implementations of the disclosure may be better understood when consideration is given to the following detailed description thereof. Such description makes reference to the annexed pictorial illustrations, schematics, graphs, drawings, and appendices. In the drawings:

FIG. 1 is a schematic diagram of an embodiment of an unmanned aircraft structure evaluation system according to the instant disclosure.

FIG. 2 is an image of an unmanned aircraft with a camera positioned about a structure of interest.

FIG. 3 is a flow chart of an exemplary embodiment of a program logic according to the instant disclosure.

FIG. 4 is an exemplary screen shot of an oblique image of the structure of interest shown in FIG. 2.

FIG. 5 is an exemplary diagram illustrating lateral and vertical offset of an unmanned aircraft in relation to a structure in accordance with the present disclosure.

FIG. 6 is an exemplary screen shot of a nadir image of the structure of interest shown in FIG. 4, the screen shot illustrating an exemplary flight plan for an unmanned aircraft.

FIG. 7 is another exemplary screen shot of nadir image of the structure shown in FIG. 6, the screen shot illustrating another exemplary flight plan for an unmanned aircraft.

FIG. 8 is an exemplary screen shot of a nadir image of the structure of interest shown in FIG. 4, the screen shot illustrating a camera path of an unmanned aircraft.

FIG. 9 is an exemplary screen shot of a structure report displayed on a display unit of a user terminal.

FIG. 10 is an exemplary screen shot of two oblique images of a structure, each oblique image showing the structure at a distinct time period.

ПОДРОБНОЕ ОПИСАНИЕ

Before explaining at least one embodiment of the inventive concept disclosed herein in detail, it is to be understood that the inventive concept is not limited in its application to the details of construction and the arrangement of the components or steps or methodologies set forth in the following description or illustrated in the drawings. The inventive concept disclosed herein is capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting in any way.

In the following detailed description of embodiments of the inventive concept, numerous specific details are set forth in order to provide a more thorough understanding of the inventive concept. It will be apparent to one of ordinary skill in the art, however, that the inventive concept within the disclosure may be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the instant disclosure.

As used herein, the terms "network-based", "cloud-based" and any variations thereof, are intended to include the provision of configurable computational resources on demand via interfacing with a computer and/or computer network, with software and/or data at least partially located on the computer and/or computer network, by pooling processing power of two or more networked processors.

As used herein, the terms "comprises", "comprising", "includes", "including", "has", "having", or any other variation thereof, are intended to be non-exclusive inclusions. For example, a process, method, article, or apparatus that comprises a set of elements is not limited to only those elements but may include other elements not expressly listed or even inherent to such process, method, article, or apparatus.

As used in the instant disclosure, the terms "provide", "providing", and variations thereof comprise displaying or providing for display a webpage (e.g., roofing webpage) to one or more user terminals interfacing with a computer and/or computer network(s) and/or allowing the one or more user terminal(s) to participate, such as by interacting with one or more mechanisms on a webpage (e.g., roofing webpage) by sending and/or receiving signals (e.g., digital, optical, and/or the like) via a computer network interface (e.g., Ethernet port, TCP/IP port, optical port, cable modem, and combinations thereof). A user may be provided with a web page in a web browser, or in a software application, for example.

As used herein, the term "structure request", "structure order", "flight plan request", "flight plan order", and any variations thereof may comprise a feature of the graphical user interface or a feature of a software application, allowing a user to indicate to a host system that the user wishes to place an order, such as by interfacing with the host system over a computer network and exchanging signals (e.g., digital, optical, and/or the like), with the host system using a network protocol, for example. Such mechanism may be implemented with computer executable code executed by one or more processors, for example, with a button, a hyperlink, an icon, a clickable symbol, and/or combinations thereof, that may be activated by a user terminal interfacing with the at least one processor over a computer network, for example.

Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by anyone of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

In addition, the use of the "a" or "an" are employed to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the inventive concept. This description should be read to include one or more, and the singular also includes the plural unless it is obvious that it is meant otherwise.

Finally, as used herein any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearance of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

Referring now to FIGS. 1 and 2, shown therein is an exemplary embodiment of an unmanned aircraft structure evaluation system 10 according to the instant disclosure. The unmanned aircraft structure evaluation system 10 comprises one or more host systems 12 interfacing and/or communicating with one or more user terminals 14 via a network 16. Generally, the one or more host systems 12 receive identification information relating to a structure of interest 21 (e.g., building) via the user terminals 14, and data indicative of the geographic positions of the structure. Using the identification information and the geographic positioning of the structure of interest 21, the one or more host systems 12 may generate unmanned aircraft information including flight path information, camera control information, and/or gimbal control information. The unmanned aircraft information may be used by an unmanned aircraft 18 to capture one or more aerial images (e.g., oblique images) of the structure of interest 21. In some embodiments, the flight path information, camera control information, and/or gimbal control information may be determined automatically by analyzing and using geo-referenced images. As such, manual manipulation and/or analysis by a user may be minimized and/or eliminated. In other embodiments, the flight path information, camera control information and/or gimbal control information may be determined with the aid of a user who supplies data by clicking on one or more displayed oblique image of the structure of interest 21 and/or otherwise inputs data into one or more of the user terminals 14.

The structure of interest 21 may be a man-made structure, such as a building. For example, in FIG. 2, the structure of interest 21 is a residential building. Alternatively, the structure may be a naturally occurring structure, such as a tree, for example.

The unmanned aircraft 18 may be any type of unmanned aerial vehicle that can be controlled by using a flight plan. Flight of the unmanned aircraft 18 may be controlled autonomously as described in further detail herein. In some embodiments, flight may be controlled using a flight plan in combination with piloting by a user located on the ground. An exemplary unmanned aircraft 18 may include the Professional SR100 UAC Camera Drone manufactured and distributed by Cadence Technology located in Singapore.

Generally, the unmanned aircraft 18 may include one or more cameras 19 configured to provide aerial images. In some embodiments, the camera 19 may be mounted on a gimbal support (e.g., three-axis gimbal). Additionally, in some embodiments, the unmanned aircraft 18 may include one or more global positioning system (GPS) receivers, one or more inertial navigation units (INU), one or more clocks, one or more gyroscopes, one or more compasses, one or more altimeters, and/or the like so that the position and orientation of the unmanned aircraft 18 at specific instances of time can be monitored, recorded and/or stored with and/or correlated with particular images.

The one or more cameras 19 may be capable of capturing images photographically and/or electronically as well as recording the time at which particular images are captured. In one embodiment, this can be accomplished by sending a signal to a processor (that receives time signals from the GPS) each time an image is captured. The one or more cameras 19 may include, but are not limited to, conventional cameras, digital cameras, digital sensors, charge-coupled devices, and/or the like. In some embodiments, one or more cameras 19 may be ultra-high resolution cameras.

The one or more cameras 19 may include known or determinable characteristics including, but not limited to, focal length, sensor size, aspect ratio, radial and other distortion terms, principal point offset, pixel pitch, alignment, and/or the like.

Referring to FIG. 1, the unmanned aircraft 18 may communicate with the one or more user terminals 14. The one or more user terminals 14 may be implemented as a personal computer, a handheld computer, a smart phone, a wearable computer, network-capable TV set, TV set-top box, a tablet, an e-book reader, a laptop computer, a desktop computer, a network-capable handheld device, a video game console, a server, a digital video recorder, a DVD-player, a Blu-Ray player and combinations thereof, for example. In an exemplary embodiment, the user terminal 14 may comprise an input unit 20, a display unit 22, a processor (not shown) capable of interfacing with the network 16, processor executable code (not shown), and a web browser capable of accessing a website and/or communicating information and/or data over a network, such as the network 16. As will be understood by persons of ordinary skill in the art, the one or more user terminals 14 may comprise one or more non-transient memories comprising processor executable code and/or software applications, for example.

The input unit 20 may be capable of receiving information input from a user and/or other processor(s), and transmitting such information to the user terminal 14 and/or to the one or more host systems 12. The input unit 20 may be implemented as a keyboard, a touchscreen, a mouse, a trackball, a microphone, a fingerprint reader, an infrared port, a slide-out keyboard, a flip-out keyboard, a cell phone, a PDA, a video game controller, a remote control, a fax machine, a network interface, and combinations thereof, for example. In some embodiments, the user terminal 14 is loaded with flight management software for controlling the unmanned aircraft 18.

The display unit 22 may output information in a form perceivable by a user and/or other processor(s). For example, the display unit 22 may be a server, a computer monitor, a screen, a touchscreen, a speaker, a website, a TV set, a smart phone, a PDA, a cell phone, a fax machine, a printer, a laptop computer, a wearable display, and/or combinations thereof. It is to be understood that in some exemplary embodiments, the input unit 20 and the display unit 22 may be implemented as a single device, such as, for example, a touchscreen or a tablet. It is to be further understood that as used herein the term user is not limited to a human being, and may comprise a computer, a server, a website, a processor, a network interface, a human, a user terminal, a virtual computer, and combinations thereof, for example.

As discussed above, the system 10 may include one or more host systems 12. The one or more host systems 12 may be partially or completely network-based or cloud based, and not necessarily located in a single physical location. Each of the host systems 12 may further be capable of interfacing and/or communicating with the one or more user terminals 14 via the network 16, such as by exchanging signals (e.g., digital, optical, and/or the like) via one or more ports (e.g., physical or virtual) using a network protocol, for example. Additionally, each host system 12 may be capable of interfacing and/or communicating with other host systems directly and/or via the network 16, such as by exchanging signals (e.g., digital, optical, and/or the like) via one or more ports.

It should be noted that multiple host systems 12 may be independently controlled by separate entities. For example, in some embodiments, system 10 may include two host systems 12 with a first host system controlled by a first company and a second host system controlled by a second company distinct from the first company.

The one or more host systems 12 may comprise one or more processors 24 working together, or independently to, execute processor executable code, one or more memories 26 capable of storing processor executable code, one or more input devices 28, and one or more output devices 30. Each element of the one or more host systems 12 may be partially or completely network-based or cloud-based, and not necessarily located in a single physical location. Additionally, in embodiments having multiple host systems 12, each host system may directly communicate with additional host systems and/or third party systems via the network 16.

The one or more processors 24 may be implemented as a single or plurality of processors 24 working together, or independently to execute the logic as described herein. Exemplary embodiments of the one or more processors 24 include a digital signal processor (DSP), a central processing unit (CPU), a field programmable gate array (FPGA), a microprocessor, a multi-core processor, and/or combinations thereof. The one or more processors 24 may be capable of communicating with the one or more memories 26 via a path (e.g., data bus). The one or more processors 24 may be capable of communicating with the input devices 28 and the output devices 30.

The one or more processors 24 may be further capable of interfacing and/or communicating with the one or more user terminals 14 and/or unmanned aircraft 18 via the network 16. For example, the one or more processors 24 may be capable of communicating via the network 16 by exchanging signals (e.g., digital, optical, and/or the like) via one or more physical or virtual ports (i.e., communication ports) using a network protocol. It is to be understood that in certain embodiments using more than one processor 24, the one or more processors 24 may be located remotely from one another, located in the same location, or comprising a unitary multi-core processor (not shown). The one or more processors 24 may be capable of reading and/or executing processor executable code and/or of creating, manipulating, altering, and/or storing computer data structures into one or more memories 26.

The one or more memories 26 may be capable of storing processor executable code. Additionally, the one or more memories 26 may be implemented as a conventional non-transient memory, such as, for example, random access memory (RAM), a CD-ROM, a hard drive, a solid state drive, a flash drive, a memory card, a DVD-ROM, a floppy disk, an optical drive, and/or combinations thereof. It is to be understood that while one or more memories 26 may be located in the same physical location as the host system 12, the one or more memories 26 may be located remotely from the host system 12, and may communicate with the one or more processor 24 via the network 16. Additionally, when more than one memory 26 is used, a first memory may be located in the same physical location as the host system 12, and additional memories 26 may be located in a remote physical location from the host system 12. The physical location(s) of the one or more memories 26 may be varied. Additionally, one or more memories 26 may be implemented as a "cloud memory" (i.e., one or more memory 26 may be partially or completely based on or accessed using the network 16).

The one or more input devices 28 may transmit data to the processors 24, and may be implemented as a keyboard, a mouse, a touchscreen, a camera, a cellular phone, a tablet, a smart phone, a PDA, a microphone, a network adapter, a wearable computer and/or combinations thereof. The input devices 28 may be located in the same physical location as the host system 12, or may be remotely located and/or partially or completely network-based.

The one or more output devices 30 may transmit information from the processor 24 to a user, such that the information may be perceived by the user. For example, the output devices 30 may be implemented as a server, a computer monitor, a cell phone, a tablet, a speaker, a website, a PDA, a fax, a printer, a projector, a laptop monitor, a wearable display and/or combinations thereof. The output device 30 may be physically co-located with the host system 12, or may be located remotely from the host system 12, and may be partially or completely network based (e.g., website). As used herein, the term "user" is not limited to a human, and may comprise a human, a computer, a host system, a smart phone, a tablet, and/or combinations thereof, for example.

The network 16 may permit bi-directional communication of information and/or data between the one or more host systems 12, the user terminals 14 and/or the unmanned aircraft 18. The network 16 may interface with the one or more host systems 12, the user terminals 14, and the unmanned aircraft 18 in a variety of ways. In some embodiments, the one or more host systems 12, the user terminals 14 and/or the unmanned aircraft 18 may communicate via a communication port. For example, the network 16 may interface by optical and/or electronic interfaces, and/or may use a plurality of network topographies and/or protocols including, but not limited to, Ethernet, TCP/IP, circuit switched paths, and/or combinations thereof. For example, the network 16 may be implemented as the World Wide Web (or Internet), a local area network (LAN), a wide area network (WAN), a metropolitan network, a wireless network, a cellular network, a GSM-network, a CDMA network, a 3G network, a 4G network, a satellite network, a radio network, an optical network, a cable network, a public switched telephone network, an Ethernet network, and/or combinations thereof. Additionally, the network 16 may use a variety of network protocols to permit bi-directional interface and/or communication of data and/or information between the one or more host systems 12, the one or more user terminals 14 and/or the unmanned aircraft 18.

In some embodiments, the one or more host systems 12, the user terminals 14, and/or the unmanned aircraft 18 may communicate by using a non-transitory computer readable medium. For example, data obtained from the user terminal 14 may be stored on a USB flash drive. The USB flash drive may be transferred to and received by the unmanned aircraft 18 thereby communicating information, such as the unmanned aircraft information including flight path information, camera control information, and/or gimbal control information from the user terminal 14 to the unmanned aircraft 18. The USB flash drive may also be used to transfer images captured by the camera 19, position, orientation and time date to the user terminal(s) 14.

Referring to FIGS. 1 and 2, the one or more memories 26 may store processor executable code and/or information comprising a structure database 32, one or more images databases 34, and program logic 36. The processor executable code may be stored as a data structure, such as a database and/or a data table, for example. In some embodiments, one or more memories of the user terminal 14 may include a structure database 32, one or more image databases 34 and program logic 36 as described in further detail herein.

The structure database 32 may include information (e.g., location, GIS data) about the structure of interest. For example, the structure database 32 may store identification information about the structure including, but not limited to, address, geographic location, latitude/longitude, and/or the like.

The one or more memories 26 may include one or more image databases 34. The one or more image databases 34 may store geo-referenced imagery. Such imagery may be represented by a single pixel map, and/or by a series of tiled pixel maps that when aggregated recreate the image pixel map. Imagery may include nadir, ortho-rectified and/or oblique geo-referenced images. The one or more processors 24 may provide the images via the image database 34 to users at the one or more user terminals 14. In some embodiments, one or more image databases 34 may be included within the user terminals 14.

The one or more memories 26 may further store processor executable code and/or instructions, which may comprise the program logic 36. The program logic 36 may comprise processor executable instructions and/or code, which when executed by the processor 24, may cause the processor 24 to execute image display and analysis software to generate, maintain, provide, and/or host a website providing one or more structure evaluation requests, for example. The program logic 36 may further cause the processor 24 to collect identification information about the structure of interest 21 (e.g., address), allow one or more users to validate a location of the structure, obtain geographical positions of the structure, and the like, as described herein.

Referring to FIG. 3, shown therein is an exemplary flow chart 40 of program logic 36 for creating a structure evaluation report according to the instant disclosure. Program logic 36 may comprise executable code, which when executed by the one or more processors 24 may cause the one or more processors 24 to execute one or more of the following steps.

In a step 42, the one or more host systems 12 may receive identification information of the structure from the user terminal 14. For example, the one or more host systems 12 may receive the address of the structure, geographic location of the structure (e.g., X, Y, Z coordinates, latitude/longitude coordinates), a location of the user terminal 14 determined by a Geographic Position System (GPS) and/or the like.

In some embodiments, the user may validate the location of the structure of interest 21. One or more processor 24 may provide one or more images via the image database 34 to the display unit 22 of the user terminal 14. For example, FIG. 4 illustrates an exemplary screen shot 60 of an oblique image 62 of the structure of interest 21 that may be displayed on the display unit 22 of the user terminal 14, shown in the block diagram of FIG. 1. The one or more images 62 may be geo-referenced images illustrating portions or all of the structure of interest 21. Referring to FIGS. 1 and 4, the program logic 36 may cause the processor 24 to provide users the one or more geo-referenced images 62 (e.g., via the display unit 22), and allow the user to validate the location of the structure of interest 21 (e.g., via the input unit 20). For example, the user may be able to use a drag-and-drop element provided by the program logic 36 via user terminal 14 to select the structure of interest 21 within the one or more geo-referenced images 62. Selection of the structure of interest 21 within the one or more geo-referenced images 62 may provide one or more validated images and a validated location of the structure of interest. It should be noted, that in some embodiments, the program logic of the user terminal 14, with or in lieu of the program logic 36 of the processor 24, may provide users the one or more geo-referenced images 62 to allow for validation of the location of the structure of interest 21.

In some embodiments, validation of the geo-referenced images may be provided by one or more additional host systems via the one or more processors 24 in lieu of, or in combination with host system 12. For example, the host system 12 may direct the user to a second host system wherein one or more processors of the second host system may provide geo-referenced images 62 from image database to the user for validation of one or more structures of interest 21.

In some embodiments, the geographic location may include coordinates, and validation of the geographic location may be provided by the user by altering one or more coordinates of the geographic location. Users may alter the one or more coordinates by methods including, but not limited to, manual manipulation, drag-and-drop elements, and the like.

In some embodiments, location of the structure of interest 21 may be automatically determined by location of the user terminal 14. For example, a user may be physically present at the structure of interest 21, and the user may be holding the user terminal 14 which determines its location using any suitable technology, such as GPS. Using location coordinates of the user terminal 14, the location of the structure of interest 21 may be determined.

In a step 44, a footprint of the structure of interest 21 may be determined. The footprint may provide a two-dimensional boundary (e.g., sides) and/or outline of the structure of interest 21. For example, the outline of the structure of interest 21 may be determined using systems and methods including, but not limited to, those described in U.S. Patent Publication No. 2010/0179787, now U.S. Pat. No. 8,145,578; U.S. Patent Publication No. 2010/0110074, now U.S. Pat. No. 8,170,840; U.S. Patent Publication No. 2010/0114537, now U.S. Pat. No. 8,209,152; U.S. Patent Publication No. 2011/0187713; U.S. Pat. No. 8,078,436; and U.S. Ser. No. 12/909,692, now U.S. Pat. No. 8,977,520; all of which are incorporated by reference herein in their entirety. In some embodiments, the footprint of the structure of interest 21 may be provided to the user via the display unit 22. For example, in some embodiments, the footprint of the structure of interest 21 may be displayed as a layer on one or more images (e.g., nadir image) via the display unit 22.

In some embodiments, the one or more processors 24 may provide, via the display unit 22, one or more websites to the user for evaluation of multiple oblique images to provide the footprint of the structure of interest 21. For example, the user and/or the processors 24 may identify edges of the structure of interest 21. Two-dimensional and/or three-dimensional information regarding the edges (e.g., position, orientation, and/or length) may be obtained from the images using user selection of points within the images and the techniques taught in U.S. Pat. No. 7,424,133, and/or stereo-photogrammetry. Using the two-dimensional and/or three-dimensional information (e.g., position orientation, and/or length), line segments may be determined with multiple line segments forming at least a portion of the footprint of the structure of interest 21.

In a step 46, data indicative of geographic positions pertaining to the footprint of the structure of interest 21 and/or structure height information may be obtained. For example, in some embodiments, the height of structure of interest 21 above the ground may be determined. The height of the structure of interest 21 above the ground may aid in determining altitude for the flight plan of the unmanned aircraft 18 as discussed in further detail herein. Measurements of the geographic positions of the structure of interest 21, such as a vertical structure, may include techniques as described in U.S. Pat. No. 7,424,133, which is hereby incorporated herein by reference in its entirety. The term "vertical structures", as used herein includes structures that have at least one portion of one surface that is not fully horizontal. For example, "vertical structures" as described herein includes structures that are fully vertical and structures that are not fully vertical, such as structures that are pitched at an angle and/or that drop into the ground. The side of a structure is not limited to only one or more walls of the structure of interest 21, but may include all visible parts of the structure of interest 21 from one viewpoint. For instance, when the present disclosure is discussing a structure of interest 21, such as a house, a "side" or "vertical side" includes the wall of the house and the roof above the wall up to the highest point on the house.

In some embodiments, more than one height may be used. For example, if the structure of interest 21 is a split-level building having a single story part and a two story part, a first height may be determined for the first story and a second height may be determined for the second story. Altitude for the flight path of the unmanned aircraft 18 may vary based on the differing heights of the structure of interest 21.

In some embodiments, using the input unit 20 and/or the display unit 22, the user may give additional details regarding geographic positions pertaining to the outline of the structure of interest 21 and/or structure height information. For example, if the structure of interest 21 is a roof of a building, the user may include identification of areas such as eaves, drip edges, ridges, and/or the like. Additionally, the user may manually give values for pitch, distance, angle, and/or the like. Alternatively, the one or more processors 24 may evaluate imagery and determine areas including eaves, drip edges, ridges and/or the like without manual input of the user.

In a step 48, using the footprint, height, and possibly additional geographic positions or information pertaining to the structure of interest 21 including the geographic location of obstructions in potential flight paths such as trees and utility wires, unmanned aircraft information may be generated by the one or more host systems 12 and/or the user terminal 14. The unmanned aircraft information may include flight path information, camera control information, and/or gimbal control information.

Flight path information may be configured to direct the unmanned aircraft 18 to fly a flight path around the structure of interest 21. In some embodiments, a flight path may be displayed to the user on one or more images (e.g., nadir, oblique) via the display unit 22. For example, FIG. 6 illustrates an exemplary screen shot 66 of a nadir image 68 showing a flight path 70 about the structure of interest 21. In some embodiments, the flight path 70 may be a displayed as a layer overlapping the nadir image 68 of the structure of interest 21 on the display unit 22 of FIG. 1.

Generally, the flight path information directs the unmanned aircraft 18 in three dimensions. Referring to FIGS. 5 and 6, the flight path information may be determined such that the flight path 70 around the structure of interest 21 is laterally and/or vertically offset from the geographic positions of the outline of the structure of interest 21. In particular, lateral offset L.sub.OFFSET and vertical offset V.sub.OFFSET may be dependent upon the height H of the structure 21, orientation of the camera relative to the unmanned aircraft 18, and characteristics of the camera 19.

Referring to FIG. 5, generally in determining offset from the structure 21, the field of view (FOV) of the camera 19 may be positioned such that a center C.sub.1 is at one half the height H of the structure 21, for example. Additionally, one or more buffer regions B may be added to the FOV. Buffer regions B may increase the angle of the FOV by a percentage. For example, buffer regions B.sub.1 and B.sub.2 illustrated in FIG. 5 may increase the angle of the FOV by 20-50%. To determine the lateral offset L.sub.OFFSET and the vertical offset V.sub.OFFSET of the camera 19 from the structure 21, a predetermined angle .THETA. within a range of 25-75 degrees may be set. Once the angle .THETA. is set, the lateral offset L.sub.OFFSET and the vertical offset V.sub.OFFSET of the camera 19 relative to the structure 21 may be determined using trigonometric principles, for example. For example, lateral offset L.sub.OFFSET may be determined based on the following equation: L.sub.OFFSET=C.sub.1*Sin(.THETA.) (EQ. 1) wherein C.sub.1 is the centerline of the field of view FOV. The vertical offset V.sub.OFFSET may be determined based on the following equation: V.sub.OFFSET=C.sub.1*Cos(.THETA.) (EQ. 2) wherein C.sub.1 is the centerline of the field of view FOV.

The flight path information may optionally direct the roll, pitch and yaw of the unmanned aircraft 18. For example, some versions of the unmanned aircraft 18 may not have a multi-axis gimbal and as such, can be directed to aim the camera 19 by changing the yaw, pitch or roll of the unmanned aircraft 18. The current yaw, pitch and roll of the unmanned aircraft 18 may be measured using a position and orientation system that is a part of the unmanned aircraft 18. In some embodiments, the position and orientation system may be implemented using microelectromechanical based accelerometers and/or microelectromechanical based gyrometers.

In many cases, there may be obstacles that lie along the flight path. Some of those obstacles may be able to be detected by the system through use of the imagery. In some embodiments, the flight path 70 may be determined such that interference with outside elements (e.g., trees and telephone wires) may be minimized. For example, FIG. 7 illustrates a variation of the flight path 70 determined in FIG. 4 wherein the flight path 70a of FIG. 7 minimizes interference by following the outline of the structure of interest 21.

A ground confidence map, as described in U.S. Pat. No. 8,588,547, which disclosure is hereby incorporated herein by reference, could be used to identify objects for which there is a high degree of confidence that the object lies elevated off of the ground. Auto-correlation and auto-aerial triangulation methods could then be used to determine the heights of these potential obstructions. If the flight path would go through one of these obstructions, it could be flagged and the algorithm could then attempt to find the best solution for getting past the obstructions: either flying closer to the structure of interest 21 as shown in FIG. 7, which might necessitate additional passes due to a finer resolution and therefore smaller path width, or by flying over the obstruction and aiming the camera 19 at a steeper oblique angle, which again may require an adjustment to the flight path to ensure full coverage. For any flight paths that are flagged for possible obstructions, a system operator could validate the corrective route chosen and alter it as necessary.

In addition to those obstacles that are identified within the image, there may also be obstacles that cannot be identified in the image. These could be newer trees or structures that were not in the original images used for flight planning, wires or other objects that may not show up in the images in enough detail to be able to determine their location, or other unexpected obstacles. As such, the unmanned aircraft 18 may also incorporate a collision detection and avoidance system in some embodiments. The collision detection and avoidance system could either be imaging based, or active sensor based. When an obstacle lies along the Flight Path, the software guiding the unmanned aircraft 18 could first attempt to move closer to the structure of interest 21 along the path from the Flight Path to the Target Path. If after a suitable threshold, which may be set at 10% of the distance (104' in the above examples, so 10% being 10.4') so that the 20% overlap still ensures complete coverage, if the unmanned aircraft 18 is unable to bypass the obstacle, the collision detection and avoidance system would steer the unmanned aircraft 18 back to its original point of collision detection and would then attempt to fly above the obstacle.

Since the software controlling the unmanned aircraft 18 keeps the camera 19 aimed at the Target Path, flying higher may still capture the necessary portions of the structure of interest 21; but the oblique down-look angle may change and the resolution may become a bit coarser. In extreme circumstances, the unmanned aircraft 18 may require operator intervention to properly negotiate around the obstacle. In these cases, the software running on a processor of the unmanned aircraft 18 would transmit a signal to the operator in the form of an audible alarm, for example, and allow the operator to steer the unmanned aircraft 18 around the obstacle. As the unmanned aircraft 18 passes the Flight Capture Points, the camera(s) 19 would fire. To ensure this, the Flight Capture Points are not just points, but may be a vertical plane that is perpendicular to the Flight Path and that passes through the Flight Capture Point. Thus, even if the unmanned aircraft 18 is 30' above or away from the Flight Path at the time, as it passes through that plane, and thus over or to the side of the Flight Capture Point, the software controlling the unmanned aircraft 18 would cause the camera 19 to fire.

The camera control information may be loaded into the software running on the processor of the unmanned aircraft 18 to control actuation of the camera 19 of the unmanned aircraft 18. For example, the camera control information may direct the camera 19 to capture images (e.g., oblique images) at one or more predefined geographic locations 74 (which are referred to herein below as Flight Capture Points), as illustrated in screen shot 72 of FIG. 8. In some embodiments, the camera control information may direct the camera 19 to capture images on a schedule (e.g., periodic, random). Further, the camera control information may control camera parameters including, but not limited to zoom, focal length, exposure control and/or the like.

The gimbal control information may be loaded into the software running on the processor of the unmanned aircraft 18 to control the direction of the camera 19 relative to the structure of interest 21. For example, the gimbal control information may control the orientation of the camera 19 in three dimensions such that during capture of an image, the camera 19 is aligned with a pre-determined location on the structure of interest 21 that are referred to below as Target Capture Points.

In a step 50, the unmanned aircraft information may be stored on one or more non-transitory computer readable medium of the host system 12 and/or user terminal 14. For example, in some embodiments, the host system 12 may determine the unmanned aircraft information, communicate the unmanned aircraft information to the user terminal 14 via the network 16, such that the unmanned aircraft information may be stored on one or more non-transitory computer readable medium. Alternatively, the user terminal 14 may determine the unmanned aircraft information and store the unmanned aircraft information on one or more non-transitory computer readable medium. In some embodiments, the one or more non-transitory computer readable medium may include a USB flash drive or other similar data storage device.

In a step 52, the unmanned aircraft information may be loaded onto the unmanned aircraft 18. For example, the unmanned aircraft information may then be loaded onto the unmanned aircraft 18 via transfer of the non-transitory computer readable medium (e.g., USB flash drive) from the user terminal 14. It should be noted that the unmanned aircraft information may be loaded and/or stored onto the unmanned aircraft 18 by any communication, including communication via the network 16.

The unmanned aircraft 18 may use the unmanned aircraft information to capture one or more oblique images of the structure of interest 21. Generally, the unmanned aircraft 18 may follow the flight path within the unmanned aircraft information obtaining the one or more oblique images as set out within the camera control information and gimbal control information. In some embodiments, a user may manually manipulate the flight path 70 of the unmanned aircraft information during flight of the unmanned aircraft 18. For example, the user may request the unmanned aircraft 18 to add an additional flight path 70 or repeat the same flight path 70 to obtain additional images.

In a step 54, the one or more processors 24 may receive one or more oblique images captured by the unmanned aircraft 18. The flight path information, camera control information and gimbal control information may direct the unmanned aircraft 18 to capture one or more oblique images at predetermined locations and times as described herein. The one or more oblique images may be communicated to the one or more processors 24 via the network and/or stored one or more non-transitory computer readable medium. The one or more oblique images may be stored in one or more image database 34. In some embodiments, the one or more oblique images may be communicated to the user terminal 14, and the user terminal 14 may communicate the images to the one or more processors 24.

In a step 56, the one or more processors 24 may generate a structure report. The program logic 36 may provide for one or more user terminals 14 interfacing with the processor 24 over the network 16 to provide one or more structure report website pages allowing users to view the structure report. For example, FIG. 9 illustrates an exemplary screen shot 76 of a structure report 78 on the display unit 22 of a user terminal 14.

One or more images 80 obtained from the camera 19 of the unmanned aircraft 18 may be used for evaluation of the structure of interest 21 for the structure report 78. For example, if the structure of interest 21 is a building, the images obtained from the camera 19 may be used in an insurance evaluation (e.g., flood damage, hail damage, tornado damage).

One or more images 80 obtained from the camera may be provided in the structure report 78. For example, the structure report 78 in FIG. 9 includes an image data set 82. The image data set 82 may include nadir and/or oblique images 80 of the structure of interest 21. Additionally, the image data set 82 may include one or more images 80 of objects of interest on and/or within the structure of interest 21. For example, if the structure report 78 details damage to a roof of the structure of interest 21, one or more images 80 of damage to the roof may be included within the image data set 82. In some embodiments, third party images of the structure of interest 21 may be included within the structure report 78.

Structural details may be provided in the structure report 78 within a structure data set 84 as illustrated in FIG. 9. The structure data set 84 may include information related to structure of interest 21 including, but not limited to, area of the structure of interest 21 (e.g., square feet), roof details (e.g., pitch, ridge length, valley length, eave length, rake length), height of the structure of interest 21, and/or the like. Additionally, the structure data set 84 may include order information for the structure report 78. For example, the structure data set 84 may include information regarding the time an order for the structure report 78 was placed, the time the order for the structure report 78 was completed, the delivery mechanism for the structure report 78, the price of the order for the structure report 78, and/or the like, for example.

Based on the flight path information, camera control information, and gimbal control information, during image capture, the location of the camera 19 relative to the structure of interest 21 for images captured may also be known. For example, in some embodiments, the X, Y, Z location (e.g., latitude, longitude, and altitude) of a location seen within each image may be determined. The information may be used to further evaluate objects on and/or within the structure of interest 21. In some embodiments, images 80 captured by the unmanned aircraft 18 may be used to generate a two or three-dimensional model of the structure of interest 21.

The unmanned aircraft structure evaluation system 10 may be used as follows.

An insurance adjustor or other field operator would arrive at the house being assessed for damage or for underwriting. He would go to an online application on a portable networked computer device (e.g., user terminal 14), such as a tablet, smart phone, or laptop, and select the property and structure of interest 21. This selection could be done with identification information, such as a GPS determining his current location, through entering a street address into the search bar, through entering the geographic location into the user terminal 14, through scrolling on a map or aerial image displayed on the user terminal 14 of the current location, or through a preselected target property made by virtually any method that results in finding the property and storing it for later retrieval.

Once the location is found, an image or 3-D Model for that property and structure of interest 21 is displayed on the screen. An oblique image, or a street side image, would provide more information to the operator for property verification as traditional orthogonal images do not include any portion of the side of the image. The 3D model (which may be textured with an oblique or street side image) would work as well. The operator verifies that the property and structure of interest 21 on the screen matches the property and structure of interest 21 that he is standing in front of to ensure that the operator generates the proper report.

The operator then clicks on the structure of interest 21 and requests a flight plan for that structure of interest 21. Software, running on either or both of the user terminal 14 and the host system 12 then isolates the structure of interest 21 and generates an outline as described above. The software also causes the user terminal 14 system to determine the height H of the structure, either by using an automated method, or by having the operator use a height tool on the oblique image, such as through the method described in U.S. Pat. No. 7,424,133. This height H is then used to automatically determine the proper flying height, lateral offset L.sub.OFFSET, and vertical offset V.sub.OFFSET offset for the flight path for the unmanned aircraft 18 (which may be an unmanned aerial system). The height H may also be used to aim the steerable camera 19 carried by the unmanned aircraft 18.

In this embodiment, first, a "Target Path" is generated that follows the path of the perimeter of the structure 21 and that is at a height over ground such that a center C.sub.1 of the field of view may be located at one-half the height of the structure of interest 21 as illustrated in FIG. 5. Thus, if it is a two-and-a-half story structure of 28' height, the Target Path would be generated such that the center C.sub.1 of the field of view may be at 14' height over ground. Although, it should be understood that the height over ground does not have to place the center C.sub.1 of the field of view to be one-half the height of the structure of interest 21 and can vary.

Next, characteristics of the camera 19 may be used, such as, for example, the desired effective resolution of the image as well as the overall sensor size of the camera 19 onboard the unmanned aircraft 18, to determine the maximum vertical swath width that may be captured on a single pass. So, for instance, if the desired effective image resolution is 1/4'' GSD, and the sensor has 4,000 pixels in the vertical orientation, then the maximum vertical swath width would be 1,000'' or 125'. A significant buffer B may be subtracted out to allow for position and orientation errors when flying, for buffeting due to wind, and for absolute position errors in the reference imagery. The size of the buffer B can vary, but can be about a 20% buffer on all sides of the imagery. As such, in this example, the maximum vertical swath width would be 75'. If the structure of interest 21 has a greater height H than this, then the structure of interest 21 may need to be captured in multiple passes. If so, using the same example numbers above, the first pass would be captured at 37.5' above ground, the second at 112.5' above ground, the third at 187.5' above ground, and so on until the entire structure of interest 21 is covered.

If the structure of interest 21 is smaller than the maximum vertical swath width, then the resolution can be increased beyond the desired effective image resolution. So in the above example of the two-and-a-half story house, the resolution could be switched to 1/8'' which would yield a maximum swath width of 37.5' which is more than sufficient to cover the 28' of structure height while still including the 20% buffer B on all sides.

Once the effective image resolution has been determined, the lateral offset L.sub.OFFSET and vertical offset V.sub.OFFSET can then be determined by calculating the path length that achieves the determined resolution. For instance, with a 5-micron sensor pitch size and a 50-mm lens, the path length would be 104'. If the desired imagery is to be captured at a .THETA. of 40-degrees (an angle from 40-degrees to 50-degrees down from horizontal is typically optimal for oblique aerial imagery) then that translates to a lateral offset L.sub.OFFSET of 79.6' stand-off distance (cosine of 40.times.104') and a vertical offset V.sub.OFFSET of 66.8' vertical height adjustment (sine of 40.times.104').

Using the Target Path as a starting point, the path would now be grown by the requisite lateral offset L.sub.OFFSET and vertical offset V.sub.OFFSET distance using standard geometry or morphological operators to create the Flight Path. For instance, if the target path were a perfect circle, the radius would be extended by the 79.6' lateral offset L.sub.OFFSET distance. If the target path were a rectangle, each side would be extended outward by the 79.6' lateral offset L.sub.OFFSET distance. The flying altitude for the Flight Path would be determined by adding the vertical offset V.sub.OFFSET distance to the height of the Target Path and then adding that to the ground elevation for the starting point of the flight path. So in the example of the 28' house, the flight altitude would be the sum of the 14' Target Path height over ground, the 66.8' vertical offset V.sub.OFFSET for the desired resolution, and the base elevation at the start, which for this example will be 280' above ellipsoid. Thus, the resulting flight height would be 360.8' above ellipsoid.

Ellipsoidal heights are used by GPS-based systems. If the elevation data available, such as an industry standard Digital Elevation Model or as the Tessellated Ground Plane information contained in the oblique images, as described in U.S. Pat. No. 7,424,133, is defined in mean sea level, the geoidal separation value for that area can be backed out to get to an ellipsoidal height, as is a well-known photogrammetric practice. From a software stand-point, a software library such as is available from Blue Marble Geo can be used to perform this conversion automatically.

Next, the software would determine Target Capture Points of the camera control information. The Target Capture Points may be spaced along the Target Path in such a manner as to ensure full coverage of the vertical structure of interest 21. This would be determined using a similar method as was done with the maximum vertical swath width. Once the desired resolution is known, it is multiplied by the number of pixels in the horizontal orientation of the sensor of the camera 19, and then sufficient overlap is subtracted. Using the above example, if there are 3,000 pixels in the sensor of the camera 19 in the horizontal orientation and the software uses the same 20% overlap and 1/8'' GSD effective image resolution that is discussed above, then a suitable spacing distance for the Target Capture Points would be 18.75'. Thus, an arbitrary start point would be selected (typically a corner along the front wall is used) and then going in an arbitrary direction, a Target Capture Point would be placed on the Target Path every 18.75' as well as one at the next corner if it occurs before a full increment. A Target Capture Point may then be placed on the start of the next segment along the Target Path and this pattern may be repeated until all the segments have Target Capture Points.

Once all the Target Capture Points have been determined, the Target Capture Points can be projected onto the Flight Path to create Flight Capture Points. This projection may be accomplished by extending a line outward from that is perpendicular to the Target Path and finding where it intersects the Flight Path. This has the effect of applying the lateral offset L.sub.OFFSET distance and vertical offset V.sub.OFFSET calculated earlier. These Flight Capture Points are then used to fire the camera 19 as the unmanned aircraft 18 passes by the Flight Capture Points. When doing so, the unmanned aircraft 18 keeps the camera aimed at the respective Target Capture Point. This aiming can be accomplished by a number of methods, such as an unmanned aircraft 18 that can turn, but is best accomplished with a computer controlled gimbal mount for the camera 19.

Alternatively, the camera 19 on the unmanned aircraft 18 could be put into "full motion video mode" whereby continuous images are captured at a high rate of speed (typically greater than 1 frame per second up to and even beyond 30 frames per second). Capturing at high frame rates ensures sufficient overlap. However, capturing at high frame rates also results in a much greater amount of image data than is needed which means longer upload times. In addition, many cameras 19 can capture higher resolution imagery in "still frame video" mode versus "full motion video" mode. But while still frame video mode is preferred from a resolution and data transfer standpoint, if the camera 19 has a full motion video mode, then the full motion video mode can also be used. When in full motion video mode, the unmanned aircraft 18 simply follows the Flight Path keeping the camera 19 aimed towards the Target Path.

The unmanned aircraft 18 would follow the indicated Flight Path through autonomous flight. There are numerous computer systems that can be configured as a flight management system to achieve this available on the market today. The flight management system, either onboard, or on the ground and communicating to the unmanned aircraft 18 through some form of remote communication, would then track the progress of the unmanned aircraft 18 along the Flight Path and each time the unmanned aircraft 18 passes a Flight Capture Point, the camera 19 would be triggered to capture a frame. Or in the event that full motion video was selected, the camera 19 would be continually firing as it flew along the Flight Path. The position and orientation of the unmanned aircraft 18 would be monitored and the camera 19 would be aimed towards the corresponding Target Capture Point, or in the event that full motion video was selected, the flight management system would keep the camera aimed towards the nearest point on the Target Path. This may be accomplished by calculating the relative directional offset between the line moving forward on the Flight Path and the line from the Flight Capture Point to the Target Capture Point (or nearest point on the Flight Path for full motion video). This then results in a yaw and declination offset for the camera gimbal. Typically, these offsets are going to be a relative yaw of 90-degrees and a relative declination equal to the oblique down-look angle selected above (in the example, 40-degrees). However, since airborne systems are continually moved around by the air, offsets for a shift in position, a shift due to crabbing, or a shift in the yaw, pitch, or roll of the unmanned aircraft 18 would need to be accounted for. Again, this may be done by using the forward path along the Flight Path that the unmanned aircraft 18 is currently on and offsetting it by the relative yaw, pitch, and roll offsets of the unmanned aircraft 18 as measured by the position and orientation system, and then further adjusted by the relative yaw and declination as described above.

Once the complete circuit of the Flight Path has been completed, the flight management system may instruct the unmanned aircraft 18 to return to its launch point and land. The operator may pull any detachable storage or otherwise transfer the imagery from the onboard storage to a removable storage system or transfer the imagery via some form of network or communications link. The resulting images may then be used by the user terminal 14 and/or the host system 12 to produce a structure and damage report. Systems for producing a structure and/or damage report are described in patents U.S. Pat. Nos. 8,078,436; 8,145,578; 8,170,840; 8,209,152; 8,401,222, and a patent application identified by U.S. Ser. No. 12/909,692, now U.S. Pat. No. 8,977,520, the entire content of each of which are hereby incorporated herein by reference. The completed report would then be provided to the operator.

In some embodiments, additional data sets may be included within the structure report 78. For example, data sets may include, but are not limited to, weather data, insurance/valuation data, census data, school district data, real estate data, and the like.

Weather data sets may be provided by one or more databases storing information associated with weather (e.g., inclement weather). A weather data set within the structure report 78 may include, but is not limited to, hail history information and/or location, wind data, severe thunderstorm data, hurricane data, tornado data, and/or the like. In some embodiments, the one or more databases providing weather information may be hosted by a separate system (e.g., LiveHailMap.com) and provide information to the host system 12.

Insurance and/or valuation data sets may be provided by one or more databases storing information associated with housing insurance and/or valuation. An insurance and/or valuation data set may include, but is not limited to, insured value of the home, insurance premium amount, type of residence (e.g., multi-family, single family), number of floors (e.g., multi-floor, single-floor), building type, and/or the like. In some embodiments, the one or more databases may be hosted by a separate system (e.g., Bluebook, MSB, 360Value) and provide information to the host system 12.

The insurance and/or valuation data set may be included within the structure report 78 and provided to the user. For example, during underwriting of a home, an insurance company may be able to request the structure report 78 on a home that is recently purchased. The information within the structure report 78 may be integrated with insurance information provided by an insurance database and used to form a quote report. The quote report may be sent to the user and/or insurance company. Alternatively, the structure report 78 may be solely sent to the insurance company with the insurance company using the information to formulate a quote.

In another example, the structure report 78 may be used in an insurance claim. In the case of a catastrophe of a customer, one or more databases may be used to provide an insurance dataset with claim information in the structure report 78. For example, an insurance database having a policy in force (PIF) and a weather database may be used to correlate information regarding an insurance claim for a particular roof. This information may be provided within the structure report 78. Additionally, in the case of loss or substantial alterations to the structure 21, multiple images may be provided within the structure report 78 showing the structure 21 at different time periods (e.g., before loss, after loss). For example, FIG. 9 illustrates an exemplary screen shot 86 of the structure 21 having with an image 88a captured at a first time period (e.g., before loss), and an image 88b captured at a second time period (e.g., after loss).

Real estate and/or census data sets may also be including within structure report 78. The real estate and/or census data sets may be provided by one or more databases having detailed information of a home. For example, a real estate data set may include, but is not limited to, the homeowner's name, the purchase price of the home, number of times the home has been on the market, the number of days the home has been on the market, the lot size, and/or the like. The census data set may include information concerning the number of residents within the home. In some embodiments, the one or more databases may be hosted by a separate system (e.g., Core Logic) and provide information to the host system 12 to provide data sets as described herein.

Other services related to structure may be provided within the structure report 78. For example, using the square footage of the roofing footprint, a price quote may be generated on the cost of insulation for the roof (e.g., energy efficiency, insulation replacement, and the like). Additionally, audits may be performed using information within one or more databases. For example, using the roofing area of a structure, historically paid insurance claims for comparables, and validation of payment for a specific claim for the home, a comparison may be made to determine whether the service payment for the specific claim was within a certain threshold. Auditing, it should be understood, may be applied to other areas as described herein as well.

Although the images of residential structures are shown herein, it should be noted that the systems and methods in the present disclosure may be applied to any residential and/or commercial building or structure. Further, the systems and methods in the present disclosure may be applied to any man-made structure and/or naturally occurring structure.

From the above description, it is clear that the inventive concept(s) disclosed herein is well adapted to carry out the objects and to attain the advantages mentioned herein as well as those inherent in the inventive concept(s) disclosed herein. While presently preferred embodiments of the inventive concept(s) disclosed herein have been described for purposed of this disclosure, it will be understood that numerous changes may be made which will readily suggest themselves to those skilled in the art and which are accomplished within the scope and spirit of the inventive concept(s) disclosed herein and defined by the appended claims.

* * * * *

Патент США №

Автор(ы)

Дата выдачи

Unmanned aircraft structure evaluation system and method

РЕФЕРАТ

Авторы:

Патентообладатель:

Заявитель:

ID семейства патентов

Номер заявки:

Дата регистрации:

Отсылочные патентные документы США

Класс патентной классификации США:

Класс совместной патентной классификации:

Класс международной патентной классификации (МПК):

Область поиска: