Выделить слова:


Патент США №	10880465
Автор(ы)	Svendsen
Дата выдачи	29 декабря 2020 г.

Determining capture instructions for drone photography based on information received from a social network

РЕФЕРАТ

Methods, Systems, and Devices are disclosed for using drone imaging to capture images at events. Capture instructions are provided to a drone device to aid in image capture. The capture instructions may include factors-of-interest. The factors-of-interest may include social graph information received from a social network and used to determine subject faces to target for image capture by the drone device. The factors-of-interest may also include current geographical location information of one or more friends identified in the social graph. This information may also be used to locate subjects for image capture. The social graph information may also be used to distribute any images captured to subjects appearing in those images.

Авторы:

Hugh Blake Svendsen (Chapel Hill, NC)

Патентообладатель:

Имя	Город	Штат	Страна	Тип
IkorongoTechnology, LLC	N/A	N/A	N/A

ID семейства патентов

74039721

Номер заявки:

16/134,122

Дата регистрации:

18 сентября 2018 г.

Отсылочные патентные документы США


Application Number	Filing Date	Patent Number	Issue Date
62561254	Sep 21, 2017
62666781	May 4, 2018

Класс патентной классификации США:	1/1
Класс совместной патентной классификации:	H04N 5/23206 (20130101); G06K 9/00979 (20130101); H04N 5/23219 (20130101); G06K 9/00288 (20130101); G06K 9/00677 (20130101); G06K 9/0063 (20130101); B64C 2201/127 (20130101); B64C 39/024 (20130101)
Класс международной патентной классификации (МПК):	H04N 7/18 (20060101); H04N 5/232 (20060101); G06K 9/00 (20060101); H04N 3/02 (20060101); B64C 39/02 (20060101)
Область поиска:	;348/144,143,61,169

Использованные источники

[Referenced By]

Патентные документы США


6526158	February 2003	Goldberg
6591068	July 2003	Dietz
6745186	June 2004	Testa et al.
6819783	November 2004	Goldberg et al.
7035440	April 2006	Kaku
7260587	August 2007	Testa et al.
7391886	June 2008	Clark et al.
7472134	December 2008	Kaku
7526106	April 2009	Clark et al.
7561723	July 2009	Goldberg et al.
7619660	November 2009	Grosvenor
7730036	June 2010	Blose et al.
7881968	February 2011	David
7953690	May 2011	Luo et al.
7965908	June 2011	Hayashi
7966223	June 2011	David
8014572	September 2011	Xiao et al.
8189880	May 2012	Wen et al.
8260674	September 2012	David
8284990	October 2012	Ma et al.
8306284	November 2012	Goldberg et al.
8458174	June 2013	Duerig
8510383	August 2013	Hurley et al.
8543454	September 2013	Fleischman et al.
8612533	December 2013	Harris et al.
8731819	May 2014	Dzubay et al.
8761523	June 2014	Cok et al.
9612599	April 2017	Bradlow
9953540	April 2018	Macfarlane
2007/0250496	October 2007	Halliday et al.
2008/0052349	February 2008	Lin
2008/0310688	December 2008	Goldberg
2010/0036875	February 2010	Miezianko et al.
2010/0103463	April 2010	Joshi et al.
2010/0191728	July 2010	Reilly et al.
2011/0022529	January 2011	Barsoba et al.
2011/0145275	June 2011	Stewart
2011/0161423	June 2011	Pratt et al.
2011/0182485	July 2011	Shochat et al.
2011/0188742	August 2011	Yu et al.
2011/0211737	September 2011	Krupka et al.
2012/0166533	June 2012	Rubinstein et al.
2012/0213404	August 2012	Steiner
2012/0221687	August 2012	Hunter et al.
2012/0250950	October 2012	Papakipos et al.
2012/0265758	October 2012	Han et al.
2012/0278387	November 2012	Garcia et al.
2012/0278395	November 2012	Garcia
2013/0117365	May 2013	Padmanabhan et al.
2013/0124508	May 2013	Paris et al.
2013/0130729	May 2013	Cok et al.
2013/0275505	October 2013	Gauglitz et al.
2014/0047027	February 2014	Moyers
2014/0122592	May 2014	Houston et al.
2014/0214843	July 2014	Arvig
2014/0316614	October 2014	Newman
2015/0019523	January 2015	Lior et al.
2016/0286244	September 2016	Chang
2016/0297545	October 2016	Yang
2016/0313736	October 2016	Schultz
2017/0064181	March 2017	Zhang et al.
2017/0070358	March 2017	Svendsen
2017/0177925	June 2017	Volkart
2017/0180623	June 2017	Lin
2017/0235308	August 2017	Gordon et al.
2017/0308076	October 2017	Tian et al.
2017/0308087	October 2017	Tian et al.
2018/0033312	February 2018	DeLuca
2018/0365839	December 2018	Feng
2019/0112048	April 2019	Culver

Зарубежные патентные документы


106791443	May 2017	CN
2009070841	Jun 2009	WO
2016100601	Jun 2016	WO
201720856	Feb 2017	WO

Другие источники

Glaser; April, DJI's new portable drone is so smart it will follow you around like a dog, "https://www.recode.net/2016/9/27/13069356/dji-portable-drone-smart-ai-se- lfies-photography", Sep. 27, 2016. cited by applicant .
Goodin; Dan, Meet Snoopy: The DIY drone that tracks your devices just about anywhere, "https://arstechnica.com/information-technology/2014/03/meet-snoopy-the-d- iy-drone-that-tracks-your-devices-just-about-anywhere/", Mar. 26, 2014. cited by applicant .
Team Commerse (No Author Attribution), Save $150 on this foldable, self-flying camera drone, "https://mashable.com/2017/11/24/hover-camera-camera-drone-foldable-black- -friday-sale/#WSNCQUOS3aqZ", Nov. 24, 2017. cited by applicant .
Erickson, On Location: Socio-locative Broadcasting as Situated Rhetorical Action, Stanford University Dissertations & Theses 3343578, 2008, 151 pages, ISBN 9780549986065, ProQuest UMI Dissertations Publishing 2008, Ann Arbor, U.S. cited by applicant.

Главный эксперт: Zhao; Daquan

Текст решения-прецедента

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Patent Applications No. 62/561,254 filed Sep. 21, 2017 and U.S. Provisional Patent Applications No. 62/666,781 filed May 4, 2018, the disclosures of which are incorporated herein by reference in their entirety.

ФОРМУЛА ИЗОБРЕТЕНИЯ

What is claimed is:

1. A server device comprising: a network interface operable to: couple the server device to a user device and a drone device over a network; and a processor and a memory coupled to the network interface and operable to: receive, from the user device, credentials enabling the server device to access social graph information from a social network server, the social graph information including friend links and corresponding geographic locations; determine, from the social graph information, capture instructions identifying one or more faces corresponding to one or more social network users identified in the social graph information; send, to the drone device, the capture instructions; receive, from the drone device, an image captured by the drone device based on the capture instructions; and send, to the user device, the image.

2. The server device of claim 1 wherein the processor and the memory further are operable to: receive drone device information identifying the drone device; and receive user device information identifying the user device.

3. The server device of claim 2 wherein the drone device information identifying the drone device includes one or more of: owner information, operator information, schedule information, current location information, capability information, and pricing policy information.

4. The server device of claim 1 further comprising: the network interface operable to: couple the server device to the user device, a drone operator device, and the drone device over the network.

5. The server device of claim 4 wherein the image received from the drone device is not relayed through the drone operator device.

6. The server device of claim 1 wherein the image received from the drone device is relayed through a drone operator device using a radio link between the drone device and the drone operator device.

7. The server device of claim 1 wherein to send the capture instructions to the drone device the processor and the memory further operable to: obtain a plurality of images associated with a sharing pool associated with an event, the user device being a participant of the sharing pool; and analyze the plurality of images to determine the capture instructions, the capture instructions including factors-of-interest including subject faces appearing in the plurality of images.

8. The server device of claim 7 wherein to send the capture instructions to the drone device the processor and the memory further operable to: determine, from the sharing pool associated with the event, geographical boundary information and temporal boundary information for the event; and include in the capture instructions the geographical boundary information and the temporal boundary information.

9. The server device of claim 1 wherein to send the capture instructions to the drone device the processor and the memory further operable to: receive, from the user device, information identifying subject faces found in a collection of images stored at the user device; and include, in the capture instructions, the information identifying the subject faces found in the collection of images stored at the user device.

10. The server device of claim 1 wherein to send the capture instructions to the drone device the processor and the memory further operable to: receive, from the user device, a second image in which one or more subject faces appear; determine information identifying the one or more subject faces found in the second image; and include, in the capture instructions, the information identifying the subject faces found in the second image.

11. The server device of claim 1 wherein to send the capture instructions to the drone device the processor and the memory further operable to: determine, from the social graph information, geographical location tracking information enabling the drone device to determine real-time locations of one or more other user devices corresponding to the one or more social network users identified from the social graph information; and include the geographical location tracking information in the capture instructions.

12. The server device of claim 11 wherein the processor and the memory are further operable to: identify addressing information for the one or more other user devices corresponding to one or more subject faces appearing in the image; and send, to the one or more other user devices, the image.

13. The server device of claim 12 wherein the processor and the memory are further operable to: receive, from the user device, first remuneration; send, in response to receiving the first remuneration from the user device, the image; receive, from a second user device, second remuneration; and send, in response to receiving the second remuneration from the second user device, the image.

14. The server device of claim 1 wherein to send the capture instructions to the drone device the processor and the memory further operable to: receive, from the user device, first factor-of-interest information and first remuneration policy information; receive, from a second user device, second factor-of-interest information and second remuneration policy information; and determine capture instructions based on the first factor-of-interest information, the first remuneration policy information, the second factor-of-interest information, and the second remuneration policy information.

15. The server device of claim 1 wherein to send the capture instructions to the drone device the processor and the memory further operable to: obtain event information for a plurality of events; determine, based on the event information for the plurality of events, event opportunity scores; designate, based on the event opportunity scores, a designated event of the plurality of events; and assign the drone device to the designated event.

16. The server device of claim 15 wherein the event information includes subject face information, geographical boundary information and temporal boundary information.

17. The server device of claim 1 wherein the processor and the memory are further operable to: receive factors-of-interest from the user device; send the factors-of-interest to the drone device; and receive the image, wherein the image was captured by the drone device based on the factors-of-interest.

18. The server device of claim 17 wherein the image received from the drone device was captured by the drone device based on the factors-of-interest including one or more of: one or more subject faces, one or more geographical areas, one or more time periods, one or more events, the social graph information, and subject tracking information.

19. The server device of claim 14 wherein the first remuneration policy information and the second remuneration policy information each identify policies including one or more of: contributing images, watching ads, purchasing prints, purchasing images, joining network, tracking information permission, and providing the social graph information.

20. The server device of claim 1 wherein receiving the image from the drone device includes receiving one or more of: a thumbnail of the image, a compressed version of the image, and a raw version of the image.

21. The server device of claim 1 wherein receiving the image from the drone device includes receiving one or more of: the image in real-time over the network, and the image in batch mode via removable media.

22. A server device comprising: a network interface operable to: couple the server device to a user device and a drone device over a network; and a processor and a memory coupled to the network interface and operable to: receive, from the user device, credentials enabling the server device to access social graph information from a social network server, the social graph information including friend links and corresponding geographic locations; determine, from the social graph information, capture instructions identifying one or more subject faces corresponding to one or more social network users identified in the social graph information; send, to the drone device, the capture instructions; receive, from the drone device, an image captured by the drone device, the one or more subject faces appearing in the image; and send, to the user device, the image.

23. A method of operating a server device comprising: receiving, from a user device, credentials enabling the server device to access social graph information from a social network server, the social graph information including friend links and corresponding geographic locations; determining, from the social graph information, capture instructions identifying one or more subject faces corresponding to one or more social network users identified in the social graph information; sending, to a drone device, the capture instructions; receiving, from the drone device, an image captured by the drone device, the one or more subject faces appearing in the image; and sending, to the user device, the image.

ОПИСАНИЕ

ОБЛАСТЬ ТЕХНИКИ

The present disclosure relates to the capture of images based on the use of unmanned aerial vehicles (drones).

УРОВЕНЬ ТЕХНИКИ

In recent years, advances in unmanned aerial vehicle (drone) technology have led to an influx of commercially available drones, the majority of which are both relatively affordable and simple to use. The use of drones has been made possible by a number of advancements, related to light weight/high strength materials, improved control and guidance systems, improved speed and reliability of electronics and communications systems, battery life, and the like. Affordable drone technology has enabled a number of new applications for drones. The use of drones in aerial photography is particularly useful in that it enables images to be taken from positions that are not easily accessible to humans.

SUMMARY OF THE DISCLOSURE

For certain applications, a single drone may service multiple users at the same time. Additionally, drones may be used to capture images in inhospitable climates and/or at inconvenient times. The present disclosure teaches how drones may be used to augment and extend the image taking capabilities of individuals. The individuals may be at events, and may be working together to capture images, or together to collectively capture images. Challenges still exist for mass utilization of drone technology, one example being the fluid state of regulatory standards for managing drone airspace.

In one aspect of the present disclosure, capture instructions are provided to a drone device by a server. The capture instructions may be explicitly provided by a user device or may be inferred implicitly by the server based on information collected from the user device. The capture instructions may include information identifying subject faces to be captured, geographical areas in which to capture images, and or temporal information indicating start and/or end times during which to capture images. The images captured by the drone device may be distributed to other devices including the user device.

It is a goal of the present disclosure to facilitate the capturing of images in situations where it is untenable using current technology by improving the functioning of the devices comprising the system including the user device, server device, drone operator device, and drone device.

In one aspect of the present disclosure, the server device receives an image captured by the drone device without making use of capture instructions provided by the server. In this aspect, the drone device may capture images under the control and guidance of the operator device. The server device may distribute the image to devices including a user device. The recipients of information identifying the image may be designated based on user devices in proximity to the image. The proximity may be one or more of geographical proximity, temporal proximity, subject face proximity, and the like.

It is a goal of the present disclosure to facilitate the sharing of images in situations where it is untenable using current technology by improving the functioning of the devices comprising the system including the user device, server device, drone operator device, and drone device.

In one aspect of the present disclosure, the server device receives information on one or more drone devices and stores that information in a drone device registry. The server device makes use of the information stored in the drone device registry to pair other devices, including user device, to a drone device for drone device service. In some respects, the server makes the designation based on information provided in a request from the user device. In some aspects, the server makes the designation by facilitating of allowing the user device to browse through the information in the drone device registry and receive inputs from the user device.

It is a goal of the present disclosure to make drone assisted imaging available at a lower cost while also increasing safety by reducing the number of drones required to provide coverage by improving the functioning of the devices comprising the system including the user device, server device, drone operator device, and drone device.

In one aspect of the present disclosure information is gathered about user interest and that interest (factors-of-interest) is used to provide instructions to the drone as to what should be captured (capture instructions). In a second aspect of the present disclosure the system may identify situations where drone photography may be appropriate, and the drone may capture images to identify users at an event, and seed and event based on that information. In a third aspect of the present disclosure, a drone device registry is provided, and a user may request service from among a plurality of drone devices, and provide control instructions to the drone device.

It is a goal of the present disclosure to improve the functioning of various electronic devices and systems to in turn improve the ability of users to obtain images captured from locations that were not previously possible. In particular, those electronic devices to be improved include the user device, server device, drone operator device, and drone device.

A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. One general aspect includes a network interface operable to: couple the server device to a user device, and an operator device over a network; and a processor and a memory coupled to the network interface and operable to. The network interface also includes send, to a drone device, capture instructions. The network interface also includes receive, from the drone device, an image captured by the drone device based on the capture instructions. The network interface also includes send, to the user device, the image. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

Implementations may include one or more of the following features. The server device where the processor and the memory further are operable to: receive drone device information identifying the drone device. The server device may also include receive user device information identifying the user device. The server device where the drone device information identifying the drone device includes one or more of: owner information, operator information, schedule information. The server device may also include current location information, capability information, pricing policy information. The server device further including: the network interface operable to. The server device may also include couple the server device to the user device, the operator device, and the drone device over the network. The server device where the image received from the drone device is not relayed through the operator device. The server device where the image received from the drone device is relayed through the operator device using a radio link between the drone device and the operator device. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: obtain a plurality of images associated with a sharing pool associated with an event, the user device being a participant of the sharing pool; and analyze the plurality of images to determine the capture instructions, the capture instructions including factors-of-interest including subject faces appearing in the plurality of images. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: determine, from the sharing pool associated with the event, geographical boundary information and temporal boundary information for the event; and include in the capture instructions the geographical boundary information and temporal boundary information. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: receiving, from the user device, information identifying subject faces found in a collection of images stored at the user device; and including, in the capture instructions, the information identifying subject faces found in the collection of images stored at the user device. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: receiving, from the user device, a second image in which one or more subject faces appear; determining information identifying the one or more subject faces found in the second image; and including, in the capture instructions, the information identifying the subject faces found in the second image. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: retrieve social graph information for a user of the user device based on credentials received from the user device; and send, to the drone device, the capture instructions, where the capture instructions include one or more subject faces corresponding to one or more social network users identified from the social graph information. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: determine, from the social graph information, geographical location tracking information enabling the drone device to determine real-time locations of one or more other user devices corresponding to the one or more social network users identified from the social graph information; and include the geographical location tracking information in the capture instructions. The server device where the processor and the memory are further operable to: identify addressing information for one or more other user devices corresponding to one or more subject faces appearing in the image; and send, to the one or more other user devices, the image. The server device where the processor and the memory are further operable to: receive, from the user device, first remuneration; send, in response to receiving the first remuneration from the user device, the image; receive, from a second user device, second remuneration; and send, in response to receiving the second remuneration from the second user device, the image. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: receive, from the user device, first factor-of-interest information and first remuneration policy information; receive, from a second user device, second factor-of-interest information and second remuneration policy information; and determine capture instructions based on first factor-of-interest information, first remuneration policy information, second factor-of-interest information, and second remuneration policy information. The server device where to send the capture instructions to the drone device the processor and the memory further operable to: obtain event information for a plurality of events; determine, based on the event information for the plurality of events, event opportunity scores; designate, based on the event opportunity scores, a designated event of the plurality of events; and assign the drone device to the designated event. The server device where the event information includes subject face information, geographical boundary information and temporal boundary information. The server device where the processor and the memory are further operable to: receive factors-of-interest from the user device; send the factors-of-interest to the drone device; and receive the image, where the image was captured by the drone device based on the factors-of-interest. The server device where the image received from the drone device was captured by the drone device based on factors-of-interest including one or more of: one or more subject faces, one or more geographical area, one or more time periods, one or more event, social graph, and subject tracking information. The server device where remuneration policies include one or more of: contribute images, watch ads, purchase prints, purchase images, join network, tracking information permission, and social graph information. The server device where receiving the image from the drone device includes receiving one or more of: a thumbnail of the image, a compressed version of the image, and a raw version of the image. The server device where receiving the image from the drone device includes receiving one or more of: the image in real-time over the network, the image in batch mode via removable media, and the image in real-time over the network. The server device where the processor and the memory are further operable to: receive a plurality of images, where the first image is one of the plurality of images; and send, to a plurality of user devices, the plurality of images, the plurality of user devices including the user device. The server device the processor and the memory are further operable to: operate a social network. The server device the processor and the memory are further operable to: communicate with a third party social network. The server device where the processor and memory are further operable to: store tracking data for the plurality of user devices; determine one or more user devices of the plurality of user devices in proximity to the first image; and send, to the one or more user devices, the first image. The server device where the processor and memory are further operable to: designate the one or more user devices of the plurality of user devices as in proximity to the first image if it was captured within a specified geographical distance and a specified temporal distance to the first image. The server device where the specified geographical distance is one of ten ft., twenty-five ft., fifty ft., one hundred ft., two hundred ft., five hundred ft., and a one thousand ft. The server device where the specified temporal distance is one of five minutes, fifteen minutes, thirty minutes, one hour, two hours, four hours, eight hours, twelve hours, and twenty four hours. The server device where the first image is not captured based on capture instructions received from the user device. The server device where the first image is captured based on capture instructions received from the user device. The server device where the first image is not captured based on control instructions received from the user device. The server device where the first image is captured based on control instructions received from the user device. The server device where the first image is not captured based on capture instructions provided by the operator device to the drone device. The server device where the first image is captured based on capture instructions provided by the operator device to the drone device. The server device where the first image is not captured based on control instructions provided by the operator device to the drone device. The server device where the first image is captured based on control instructions provided by the operator device to the drone device. The server device where the processor and memory are further operable to: receive, from the operator device, a distribution list of user devices; and send, to the distribution list of user devices, the first image. The server device where the processor and memory are further operable to: receive, from the user device, a distribution list of user devices; and send, to the distribution list of user devices, the first image. The server device where the processor and memory are further operable to: receive, from the user device, information, the information enabling access to a social network graph; and determine, based on the information, a distribution list of user devices send, to the distribution list of user devices, the first image. The server device where the processor and memory are further operable to: associate the first image with a sharing pool; and send, to participants of the sharing pool, the first image. The server device where the processor and memory are further operable to: initiate, based on a request from the user device, a sharing pool. The server device where the processor and memory are further operable to: initiate, based on a request from the user device, a sharing pool. The server device where the processor and memory are further operable to: receive boundary information, the boundary information identifying geographical boundaries of a geo-fence. The server device may also include in response to receiving the first image, determine user devices currently within the geo-fence. The server device may also include send, to the user devices currently within the geo-fence, the first image. The server device where to designate the available drone device the processor and the memory are further operable to: send, to the requesting user device, drone device information for ones of the plurality of drone devices; and receive, from the requesting user device, information indicating the available drone device from the ones of the plurality of drone devices. The server device where the processor and the memory are further operable to: send the drone device information in one of xml and html format. The server device where to designate the available drone device the processor and the memory are further operable to: perform a comparison, by the server device, of the request to the drone device information; and designate, based on the comparison, the available drone device of the plurality of drone devices. The server device where the request includes a current geographic location of the requesting user device and the drone device information includes a current geographic location of the available drone device, the processor and memory further operable to: compare the current geographic location of the requesting user device to the current geographic location of the available drone device. The server device where the processor and the memory are further operable to: add the available drone device to a drone device repository. The server device where the processor and the memory are further operable to: receive, from the requesting user device, control instructions. The server device may also include send, to the designated drone device, the control instructions. The server device where the processor and the memory are further operable to: receive, from the designated drone device, an image captured by the designated drone device based on the control instructions. The server device may also include send, to the requesting user device, the image. The server device where the image received from the designated drone device is relayed through an operator device using a radio link between the designated drone device and the operator device. The server device where the image received from the designated drone device is tagged with information including: altitude of capture. The server device may also include direction of capture. The server device may also include time of capture. The server device may also include date of capture. The server device may also include operator information. The server device may also include owner information. The server device where the processor and the memory are further operable to: receive, from the designated drone device, an image captured by the designated drone device based on the control instructions; store the image at the server; and send, to the requesting user device, information identifying the image. The server device further including: the network interface further operable to. The server device may also include couple the server device to the designated drone device over the network; and the processor and the memory are further operable to. The server device may also include receive the image from the designated drone device. The server device where the image received from the designated drone device is not relayed through an operator device. The server device where the processor and the memory are further operable to: receive, from the requesting user device, tracking instructions; and send, to the designated drone device, the tracking instructions, where the tracking instructions enable the designated drone device to follow a reference position user device as it changes geographic location over time. The server device where capture capabilities include capture resolution, frame rate, and media format. The server device where the request includes start time, geographic location information, and capture capabilities. The server device where drone device information includes communication connection type, communication connection bandwidth, battery life, maximum altitude, minimum altitude, coverage area, flight speed, availability, first information identifying one or more operators, second information identifying one or more owners. The server device where the drone device information identifying the registering drone device includes one or more of: owner information, operator information, schedule information. The server device may also include current location information, capability information, pricing policy information. The server device where the processor and the memory are further operable to: send, to the available drone device, a scheduling request, the scheduling request identifying a time and geographic location. The server device may also include receive, from the available drone device, scheduling confirmation. The server device where in response to receiving the request the processor and the memory are further operable to: send, to user device, a request acknowledgement; commence drone coverage; and send, to user device, commencement acknowledgement. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

One general aspect includes a server device including: a network interface operable to. The server device also includes couple the server device to a user device, and an operator device over a network; and a processor and a memory coupled to the network interface and operable to. The server device also includes receive, from a drone device, a first image. The server device also includes designate the user device to receive the first image. The server device also includes send, to the user device, the first image. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

Implementations may include one or more of the following features. The server device where the processor and the memory are further operable to: receive a plurality of images, where the first image is one of the plurality of images; and send, to a plurality of user devices, the plurality of images, the plurality of user devices including the user device. The server device the processor and the memory are further operable to: operate a social network. The server device the processor and the memory are further operable to: communicate with a third party social network. The server device where the processor and memory are further operable to: store tracking data for the plurality of user devices; determine one or more user devices of the plurality of user devices in proximity to the first image; and send, to the one or more user devices, the first image. The server device where the processor and memory are further operable to: designate the one or more user devices of the plurality of user devices as in proximity to the first image if it was captured within a specified geographical distance and a specified temporal distance to the first image. The server device where the specified geographical distance is one of ten ft., twenty-five ft., fifty ft., one hundred ft., two hundred ft., five hundred ft., and a one thousand ft. The server device where the specified temporal distance is one of five minutes, fifteen minutes, thirty minutes, one hour, two hours, four hours, eight hours, twelve hours, and twenty four hours. The server device where the first image is not captured based on capture instructions received from the user device. The server device where the first image is captured based on capture instructions received from the user device. The server device where the first image is not captured based on control instructions received from the user device. The server device where the first image is captured based on control instructions received from the user device. The server device where the first image is not captured based on capture instructions provided by the operator device to the drone device. The server device where the first image is captured based on capture instructions provided by the operator device to the drone device. The server device where the first image is not captured based on control instructions provided by the operator device to the drone device. The server device where the first image is captured based on control instructions provided by the operator device to the drone device. The server device where the processor and memory are further operable to: receive, from the operator device, a distribution list of user devices; and send, to the distribution list of user devices, the first image. The server device where the processor and memory are further operable to: receive, from the user device, a distribution list of user devices; and send, to the distribution list of user devices, the first image. The server device where the processor and memory are further operable to: receive, from the user device, information, the information enabling access to a social network graph; and determine, based on the information, a distribution list of user devices send, to the distribution list of user devices, the first image. The server device where the processor and memory are further operable to: associate the first image with a sharing pool; and send, to participants of the sharing pool, the first image. The server device where the processor and memory are further operable to: initiate, based on a request from the user device, a sharing pool. The server device where the processor and memory are further operable to: initiate, based on a request from the user device, a sharing pool. The server device where the processor and memory are further operable to: receive boundary information, the boundary information identifying geographical boundaries of a geo-fence. The server device may also include in response to receiving the first image, determine user devices currently within the geo-fence. The server device may also include send, to the user devices currently within the geo-fence, the first image. The server device where to designate the available drone device the processor and the memory are further operable to: send, to the requesting user device, drone device information for ones of the plurality of drone devices; and receive, from the requesting user device, information indicating the available drone device from the ones of the plurality of drone devices. The server device where the processor and the memory are further operable to: send the drone device information in one of xml and html format. The server device where to designate the available drone device the processor and the memory are further operable to: perform a comparison, by the server device, of the request to the drone device information; and designate, based on the comparison, the available drone device of the plurality of drone devices. The server device where the request includes a current geographic location of the requesting user device and the drone device information includes a current geographic location of the available drone device, the processor and memory further operable to: compare the current geographic location of the requesting user device to the current geographic location of the available drone device. The server device where the processor and the memory are further operable to: add the available drone device to a drone device repository. The server device where the processor and the memory are further operable to: receive, from the requesting user device, control instructions. The server device may also include send, to the designated drone device, the control instructions. The server device where the processor and the memory are further operable to: receive, from the designated drone device, an image captured by the designated drone device based on the control instructions. The server device may also include send, to the requesting user device, the image. The server device where the image received from the designated drone device is relayed through an operator device using a radio link between the designated drone device and the operator device. The server device where the image received from the designated drone device is tagged with information including: altitude of capture. The server device may also include direction of capture. The server device may also include time of capture. The server device may also include date of capture. The server device may also include operator information. The server device may also include owner information. The server device where the processor and the memory are further operable to: receive, from the designated drone device, an image captured by the designated drone device based on the control instructions; store the image at the server; and send, to the requesting user device, information identifying the image. The server device further including: the network interface further operable to. The server device may also include couple the server device to the designated drone device over the network; and the processor and the memory are further operable to. The server device may also include receive the image from the designated drone device. The server device where the image received from the designated drone device is not relayed through an operator device. The server device where the processor and the memory are further operable to: receive, from the requesting user device, tracking instructions; and send, to the designated drone device, the tracking instructions, where the tracking instructions enable the designated drone device to follow a reference position user device as it changes geographic location over time. The server device where capture capabilities include capture resolution, frame rate, and media format. The server device where the request includes start time, geographic location information, and capture capabilities. The server device where drone device information includes communication connection type, communication connection bandwidth, battery life, maximum altitude, minimum altitude, coverage area, flight speed, availability, first information identifying one or more operators, second information identifying one or more owners. The server device where the drone device information identifying the registering drone device includes one or more of: owner information, operator information, schedule information. The server device may also include current location information, capability information, pricing policy information. The server device where the processor and the memory are further operable to: send, to the available drone device, a scheduling request, the scheduling request identifying a time and geographic location. The server device may also include receive, from the available drone device, scheduling confirmation. The server device where in response to receiving the request the processor and the memory are further operable to: send, to user device, a request acknowledgement; commence drone coverage; and send, to user device, commencement acknowledgement. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

One general aspect includes a server device including: a network interface operable to. The server device also includes couple the server device to a plurality of user devices and a plurality of operator devices over a network; and a processor and a memory coupled to the network interface and operable to: for a plurality of drone devices. The server device also includes receive drone device information for a registering drone device of the plurality of drone devices. The server device also includes register, at the server device, the registering drone device of the plurality of drone devices based on the drone device information. The server device also includes receive, from a requesting user device of the plurality of user devices, a request for drone service from an available drone device of the plurality of drone devices. The server device also includes designate the available drone device of the plurality of drone devices as a designated drone device of the plurality of drone devices. The server device also includes assign the designated drone device of the plurality of drone devices to provide drone service to the requesting user device. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

Implementations may include one or more of the following features. The server device where to designate the available drone device the processor and the memory are further operable to: send, to the requesting user device, drone device information for ones of the plurality of drone devices; and receive, from the requesting user device, information indicating the available drone device from the ones of the plurality of drone devices. The server device where the processor and the memory are further operable to: send the drone device information in one of xml and html format. The server device where to designate the available drone device the processor and the memory are further operable to: perform a comparison, by the server device, of the request to the drone device information; and designate, based on the comparison, the available drone device of the plurality of drone devices. The server device where the request includes a current geographic location of the requesting user device and the drone device information includes a current geographic location of the available drone device, the processor and memory further operable to: compare the current geographic location of the requesting user device to the current geographic location of the available drone device. The server device where the processor and the memory are further operable to: add the available drone device to a drone device repository. The server device where the processor and the memory are further operable to: receive, from the requesting user device, control instructions. The server device may also include send, to the designated drone device, the control instructions. The server device where the processor and the memory are further operable to: receive, from the designated drone device, an image captured by the designated drone device based on the control instructions. The server device may also include send, to the requesting user device, the image. The server device where the image received from the designated drone device is relayed through an operator device using a radio link between the designated drone device and the operator device. The server device where the image received from the designated drone device is tagged with information including: altitude of capture. The server device may also include direction of capture. The server device may also include time of capture. The server device may also include date of capture. The server device may also include operator information. The server device may also include owner information. The server device where the processor and the memory are further operable to: receive, from the designated drone device, an image captured by the designated drone device based on the control instructions; store the image at the server; and send, to the requesting user device, information identifying the image. The server device further including: the network interface further operable to. The server device may also include couple the server device to the designated drone device over the network; and the processor and the memory are further operable to. The server device may also include receive the image from the designated drone device. The server device where the image received from the designated drone device is not relayed through an operator device. The server device where the processor and the memory are further operable to: receive, from the requesting user device, tracking instructions; and send, to the designated drone device, the tracking instructions, where the tracking instructions enable the designated drone device to follow a reference position user device as it changes geographic location over time. The server device where capture capabilities include capture resolution, frame rate, and media format. The server device where the request includes start time, geographic location information, and capture capabilities. The server device where drone device information includes communication connection type, communication connection bandwidth, battery life, maximum altitude, minimum altitude, coverage area, flight speed, availability, first information identifying one or more operators, second information identifying one or more owners. The server device where the drone device information identifying the registering drone device includes one or more of: owner information, operator information, schedule information. The server device may also include current location information, capability information, pricing policy information. The server device where the processor and the memory are further operable to: send, to the available drone device, a scheduling request, the scheduling request identifying a time and geographic location. The server device may also include receive, from the available drone device, scheduling confirmation. The server device where in response to receiving the request the processor and the memory are further operable to: send, to user device, a request acknowledgement; commence drone coverage; and send, to user device, commencement acknowledgement. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

One general aspect includes a server device including: a network interface operable to. The server device also includes couple the server device to a plurality of user devices and a plurality of operator devices over a network; and a processor and a memory coupled to the network interface and operable to. The server device also includes register a plurality of drone devices; receive, from a first user device of the plurality of user devices, a request for drone service from a first drone device of the plurality of drone devices; and assign a designated drone device of the plurality of drone devices to provide drone service to the first user device. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure, and together with the description serve to explain the principles of the disclosure.

FIG. 1A illustrates an exemplary system for drone assisted image capture;

FIG. 1B illustrates a block diagram for an exemplary social network server;

FIG. 1C illustrates an exemplary data structure for representing drone device information;

FIG. 1D illustrates an exemplary data structure for representing control instructions;

FIG. 1E illustrates an exemplary data structure for representing capture instructions;

FIG. 1F illustrates an exemplary data structure for representing factor-of-interest information;

FIG. 1G illustrates an exemplary data structure for representing remuneration policy information;

FIG. 1H illustrates an exemplary data structure for representing image metadata;

FIG. 2A illustrates a network diagram for operating the system of FIG. 1A and describes the transfer of image data a radio link;

FIG. 2B illustrates a network diagram for operating the system of FIG. 1A and describes the transfer of image data over a network;

FIG. 2C illustrates a network diagram for operating the system of FIG. 1A and describes the transfer of image data employing a removal storage medium;

FIG. 3 illustrates a network diagram for operating of drone imaging where the drone device captures images based on the faces appearing in images contained in a sharing pool;

FIG. 4 illustrates a network diagram for operating drone imaging where the drone device captures images based on information stored in a social network;

FIG. 5A illustrates a network diagram for operating drone imaging where the server device determines image capture priorities for the drone device based on requests received from user devices;

FIG. 5B illustrates an exemplary computation for determining capture priority scores;

FIG. 6A illustrates a network diagram for operating drone imaging where a drone is assigned to an event determined from a plurality of events based on event opportunity scores;

FIG. 6B illustrates an exemplary computation for determining opportunity scores;

FIG. 7 illustrates a network diagram for operating drone imaging where the server device distributes captured images based on information derived from the captured images;

FIG. 8 illustrates a network diagram for operating drone imaging where a plurality of drones register with the server device and a user device is operable to receive control of one of the drones;

FIG. 9 is an aerial view illustrating the use of drone imaging at a sporting event;

FIG. 10 is an aerial view illustrating the use of drone imaging at a vacation event;

FIG. 11 is an aerial view illustrating the use of drone imaging at a parade event;

FIG. 12 is an aerial view illustrating the use of drone imaging at a landmark viewing event;

FIG. 13 is an aerial view illustrating the use of drone imaging at an landmark viewing event;

FIG. 14 is a side view of an exemplary drone device;

FIG. 15 is a block diagram of the drone operator device of FIGS. 1A, 1B and 1C according to some aspects of the present disclosure;

FIG. 16 is a block diagram of the drone device of FIGS. 1A, 1B and 1C according to some aspects of the present disclosure; and

FIG. 17 is a block diagram of the server device of FIGS. 1A, 1B and 1C according to some aspects of the present disclosure.

ПОДРОБНОЕ ОПИСАНИЕ

The present disclosure is described with specificity to meet statutory requirements.

However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or elements similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the term "step" may be used herein to connote different aspects of methods employed, the term should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.

As referred to herein, the term "user device" and "drone operator device" should be broadly construed. It can include any type of mobile device, for example, a smart phone, a cell phone, a pager, a personal digital assistant (PDA, e.g., with GPRS NIC), a mobile computer with a cellular radio, or the like. A typical mobile device is a wireless data access-enabled device (e.g., an iPHONE.RTM. smart phone, a BLACKBERRY.RTM. smart phone, a NEXUS ONE.TM. smart phone, an iPAD.TM. device, or the like) that is capable of sending and receiving data in a wireless manner using protocols like the Internet Protocol, or IP, and the wireless application protocol, or WAP. This allows users to access information via wireless devices, such as smart phones, mobile phones, pagers, two-way radios, communicators, and the like. Wireless data access is supported by many wireless networks, including, but not limited to, CDPD, CDMA, GSM, PDC, PHS, TDMA, FLEX, ReFLEX, iDEN, TETRA, DECT, DataTAC, Mobitex, EDGE and other 2G, 3G, 4G and LTE technologies, and it operates with many handheld device operating systems, such as PalmOS, EPOC, Windows CE, FLEXOS, OS/9, JavaOS, iOS and Android. Typically, these devices use graphical displays and can access the Internet (or other communications network) on so-called mini- or micro-browsers, which are web browsers with small file sizes that can accommodate the reduced memory constraints of wireless networks. In a representative embodiment, the mobile device is a cellular telephone or smart phone that operates over GPRS (General Packet Radio Services), which is a data technology for GSM networks. In addition to a conventional voice communication, a given mobile device can communicate with another such device via many different types of message transfer techniques, including SMS (short message service), enhanced SMS (EMS), multi-media message (MMS), email WAP, paging, or other known or later-developed wireless data formats. Although many of the examples provided herein are implemented on a mobile device, the examples may similarly be implemented on any suitable device with the necessary characteristics and capabilities.

Throughout this specification, like reference numbers signify the same elements throughout the description of the figures.

When elements are referred to as being "connected" or "coupled," the elements can be directly connected or coupled together or one or more intervening elements may also be present. In contrast, when elements are referred to as being "directly connected" or "directly coupled," there are no intervening elements present.

The subject matter may be embodied as devices, systems, methods, and/or computer program products. Accordingly, some or all of the subject matter may be embodied in hardware and/or in software (including firmware, resident software, micro-code, state machines, gate arrays, etc.) Furthermore, the subject matter may take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system. In the context of this document, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

The computer-usable or computer-readable medium may be for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media.

Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and may be accessed by an instruction execution system. Note that the computer-usable or computer-readable medium can be paper or other suitable medium upon which the program is printed, as the program can be electronically captured via, for instance, optical scanning of the paper or other suitable medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" can be defined as a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above-mentioned should also be included within the scope of computer-readable media.

When the subject matter is embodied in the general context of computer-executable instructions, the embodiment may comprise program modules, executed by one or more systems, computers, or other devices. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

Operating environments in which embodiments of the present disclosure may be implemented are also well-known. In a representative embodiment, a user device 20 (shown in FIG. 1A for example), such as a mobile device, is connectable to a transmission functionality that varies depending on implementation. Thus, for example, where the operating environment is a wide area wireless network (e.g., a 2.5G network, a 3G network, a 4G network, or a 5G network), the transmission functionality comprises one or more components such as a mobile switching center (MSC) (an enhanced ISDN switch that is responsible for call handling of mobile subscribers), a visitor location register (VLR) (an intelligent database that stores on a temporary basis data required to handle calls set up or received by mobile devices registered with the VLR), a home location register (HLR) (an intelligent database responsible for management of each subscriber's records), one or more base stations (which provide radio coverage with a cell), a base station controller (BSC) (a switch that acts as a local concentrator of traffic and provides local switching to effect handover between base stations), and a packet control unit (PCU) (a device that separates data traffic coming from a mobile device). The HLR also controls certain services associated with incoming calls. Of course, the present disclosure may be implemented in other and next-generation mobile networks and devices as well. The mobile device is the physical equipment used by the end user, usually a subscriber to the wireless network. Typically, a mobile device is a 2.5G-compliant device, 3G-compliant device, 4G-compliant device, or 5G-compliant device) that includes a subscriber identity module (SIM), which is a smart card that carries subscriber-specific information, mobile equipment (e.g., radio and associated signal processing devices), a user interface (or a man-machine interface (MMI), and one or more interfaces to external devices (e.g., computers, PDAs, and the like). The mobile device may also include a memory or data store.

FIG. 1A shows a system diagram for operating a drone according to some embodiments. In these embodiments, the system is comprised of a user device 20, drone device 40, a server device 60, drone operator device 80 and a network 15. The drone device 40 and the drone operator device 80 also communicate over a separate radio link 16. A removable storage medium 18 may be used to move images from the drone device 40 to the drone operator device 80.

The drone device communicates to the rest of the system through three channels, a network connection 15--which is often implemented as TCP/IP, a radio link 16, and a removable storage medium 18. The three channels may be used in any combination, but one of either the network connection 15 or radio link 16 must be present.

The user device 20 is comprised of a control system 21, UI module 22, communication module 23, imaging module 24 and configuration module 25. The control system 21 is described in FIG. 15. The UI module 22 operates to facilitate interactions between the user of the user device 20 and the hardware and software of the user device 20. The communication module 23 facilitates communications between the user device 20 and other devices connected through the network 15, such as the drone device 40 and the server device 60. The imaging module 24 handles the capture and storage of images. The configuration module 25 facilitates configurations of the user device in some embodiments of the present disclosure. The user device 20 is operated by the user 10.

The drone device 40 is comprised of a control system 41, guidance module 42, communication module 43, power management module 44, capture module 45 storage module 46 and a synchronization module 47. The control system 41 is described in FIG. 16. The guidance module 42 operates to fly the drone device in a plurality of operating modes. The communication module 43 provides one or more communication channels for the drone device including wide area communication over the Internet, IP connectivity over local area networks, and RF communications between the drone device and the drone operator device 80. The power management module 44 monitors and controls the power usage and battery level of the battery supplying power to the drone device 40. The capture module 45 handles the capture and storage of images at the drone device 40. The storage module 46 handles the storage of images captured at the drone device 40. The synchronization module 47 handles the transmission of reduced resolution and full resolution images from the drone device 40 to the server device 60 either directly or indirectly through the drone operator device 80.

The server device 60 is comprised of a control system 61, event management module 62, user management module 63, drone management module 64, image analysis engine 65 and invitation management module 66. The control system 61 is described in FIG. 17. The event management module 62 operates to store information related to one or more image sharing events in the event registry 100. Information stored in the event registry for each event may include associated sharing pool information wherein the sharing pool information may include pool name, pool id, pool type, pool description, pool moderator, pool invitees, pool participants, pool subjects, pool initiation policies, pool termination policies, pool notifications, pool images, and pool tags. Image sharing events and sharing pools are further described in U.S. patent application Ser. No. 14/574,513, entitled METHODS FOR SHARING IMAGES CAPTURED AT AN EVENT, filed on Dec. 18, 2014, currently pending, which is hereby incorporated by reference in its entirety. As used herein, the term "event" refers to the real-world gathering of individuals characterized by geographic location, time, and participants among other things. As used herein, the term "sharing pool refers" to the electronic embodiment of an event including amount other things the fields and parameters characterizing the event.

The user management module 63 operates to store information related to one or more users in the user registry 102. Information stored in the user registry for each user may include: profile information, user devices associated with the user, credentials for interfacing with one or more social networks with which the user has an account, images captured by the user, factors of interest associated with the user, etc. The operator management module 64 operates to store information related to one or more drone operators in the drone operator registry 106. Information stored in the drone operator registry 106 for each drone operator may include: name, geographic location, availability schedule, associated drone devices, experience level, credentials, etc. The drone management module 65 operates to store information related to one or more drone devices in the drone device registry 106. Information stored in the drone device registry 106 for each drone device are described in FIG. 1C. The image analysis engine 66 operates to provide image functions such as compression, filtering, storage, auto correction, tagging, facial recognition, object recognition, etc. The invitation management module 67 handles the creation, transmission, and acknowledgement reception of invitations to events. In various embodiments, the aforementioned registries (100, 102, 104, 106) may be stored as xml in the file system, records in a database, elements in a blockchain, or any combination thereof.

The drone operator device 80 is operated by the drone operator 12. The drone operator 12 launches and provides control of the drone device 40 while the drone device 40 is being manually positioned.

Those of ordinary skill in the art will appreciate that the network 15 is not limited by the implementations listed above. More specifically, the network 15 may be any type of network suitable to allow interaction between the user devices 20, drone operator devices 80 and the server devices 60. For example, the network 15 may be a wired network, a wireless network, or any combination thereof. Further, the network 15 may include a distributed computing network, an intranet, a local-area network (LAN) and/or a wide-area network (WAN), or any combination thereof.

The removable storage medium 18 may be one or more of a Compact Flash card, Secure Digital card, Memory Stick, and the like.

In some embodiments, the server device 60 may communicate with the drone operator device through the network and the drone operator device 80 communicates with the drone device 40 using the radio link 16. In some embodiments, the drone operator device 80 may serve as a pass through to pass data or commands back and forth from the server device 60 to the drone device 40.

As used herein, the term "social network" refers to a server device that enables client devices associated with users to create and store electronic friend relationship information. Those friend relationships may be symmetric in that one user invites another user to connect (or link), and the other user must accept the electronic invitation before the symmetric friend relationship is created and stored by the server device. The friend relationships may also be asymmetric in that one user may request to follow another user, and the other user need not accept before the asymmetric friend relationship is created and stored by the server device. In some embodiments, the server device may be operable to support both symmetric and asymmetric friend relationships. Examples of server devices that should not be considered social networks are e-mail systems and trust networks. With e-mail, all you need is someone e-mail address to be able to communicate with them and friending is not required. Trust networks typically operate on inference engines where trust is inferred by actions taken by the various users who need not be connected as friends. A server device may be both a social network and a trust network, but just by being one, does not automatically make it the other. An example of a trust network is a news site that enables visitors to comment on articles. Visitors that often contribute valuable comments are awarded a high trust rating. Visitors who contribute off topic comments laced with profanity are awarded a low trust rating.

As used herein, the term "social graph" refers to the electronic friend connections stored by the server device. In some embodiments, this information may be available for export by the server device, such as is the case with Facebook Connect.

As used herein, the term "social distance" refers to the number of hops in the social graph to get from one user to another user. So, the social distance between two friends is one. The social distance between a user and a friend of a friend of a friend is three.

Referring now to FIG. 1B, a block diagram for an exemplary social network server 130. The exemplary social network server 130 is comprised of invitation/follow handler 131, graph export handler 132 and user accounts 133. The invitation/follow handler 131 facilitates the invitation and acceptance signaling required for symmetric friend link 135 creation as well as the signaling required for asymmetric friend link 136 creation. The graph export handler 132 facilitates access to social graph information 134 by external entities. The social network server 130 may also store location history information 137 including facilitating access to user's current geographical location. The user's current geographical location is typically automatically determined and periodically updated without user action beyond possible initial configuration input. However, in some cases, when automatic geographical location cannot be determined, the user may enter this information in directly, typically by choosing a point on a graphical map displayed on the user device 20.

In some embodiments, the social network server 130 is deployed separately and controlled by another entity distinct from the server device 60, and the social network server provides services and data to the server device to achieve the desired results. In some embodiments, the social network server 130 and the server device 60 are operated together by the same entity.

FIG. 1C illustrates an exemplary data structure for representing drone device information 140. This information is provided by the drone device 40 and/or the drone operator device 80. It is stored in the drone device registry 106, and may be updated periodically.

TABLE-US-00001 DRONE DEVICE INFORMATION DESCRIPTION Drone device A unique machine identifier for the drone device. identifier 141 Owner information Identifies one or more owners of the drone device. 142 This may or may not be the same as the drone operator 12. Operator information Identifies one or more drone operators of the drone 143 device. Schedule information Identifies time periods during which the drone is 144 already scheduled, time periods for when the drone device is available, and time periods where the drone device is not available for scheduling. Current location 145 Identifies the current geographical location for the drone device if the status of the drone device indicates that it is currently operating. Capture capability Identifies items such as the capture resolution, 146 capture rate, capture format, etc. Pricing information Identifies remuneration types accepted, 147 corresponding rates, and services available Restrictions 148 Identifies flight restrictions such as maximum height, min height, coverage range, maximum flight time, etc. Status 149 Identifies the current status of the drone device, such as operating and offline.

FIG. 1D illustrates an exemplary data structure for representing drone control instructions 150. In some embodiments of the present disclosure, a user device 20 may control a drone device 40 directly and send control instructions 150 to the drone device 40 through the server device. In some embodiments, the control instructions 150 are transmitted through the drone operator device 80 and may be modified by the drone operator device 80 prior to delivery to the drone device 40. For example, as a safety precaution, if the user device 20 is attempting to operate the drone device 40 in an unsafe manner (such as too close to the event action or too close to a landmark), then the drone operator device 80 may take control and/or alter/or override the control instructions 150.

TABLE-US-00002 CONTROL INSTRUCTIONS DESCRIPTION Position hold 151 Instruct the drone device to hold current GPS position. Return to Instruct drone device to return to the GPS location home 152 associated with the drone's home location. Autonomous Instruct drone device to proceed along a provided flight 153 series of GPS waypoints. Track subject 154 Instruct drone device to tract a certain subject (face or tracking info). Capture 155 Instruct drone device to take a picture. Position Instruct drone device to move to a specified location. camera 156

FIG. 1E illustrates an exemplary data structure for representing capture instructions 160.

TABLE-US-00003 CAPTURE INSTRUCTIONS DESCRIPTION Factors-of-interest 170 Refers to the factors-of-interest 170 structure below. Remuneration policy 180 Refers to the remuneration policy information 180 structure below.

FIG. 1F illustrates an exemplary data structure for representing factor-of-interest information 170. This information may be explicitly received via user input at a user device 20, or implicitly inferred from user behaviors, preferences, geographic location history, etc. For example, in some embodiments, a user device 20 will take an image comprising factors-of-interest and send it to the server device 60. Based on the received image, the server device 60 will isolate the factors-of-interest and send them to the drone device 40. In some embodiments, the user device 20 will determine the factors-of-interest and send them to the server device 40. In some embodiments, the user device will enable the user of the user device 20 to input the factors-of-interest explicitly, and those will be sent to the server device drone device 40.

TABLE-US-00004 FACTOR OF INTEREST DESCRIPTION Subject faces 171 Information identifying the face of a subject, a subject being any person including a drone device operator 12 or user device owner 10. This information may be used by the drone to employ facial recognition to identify and track users. Geographical Information identifying a geographical area. For areas 172 example, the boundaries of an event. This information may be used by the drone for positioning purposes. Times period 173 Information identifying a temporal period. For example, the start and end times of an event. This information may be used by the drone for determining when to take images and for scheduling purposes. Events 174 Information identifying a planned public or social occasion such as a sporting event, concert event and the like. Social graph 175 Information available from a social network (social graph information 134), for example through Facebook Connect. This information may be used to extract Subject Face 171 information for one or more friends of a user, which in turn may be provided to the drone for use in facial recognition and user tracking. Subject tracking Information allowing the geographical location information 176 tracking of a subject. This may include the current geographical location of a user. This is used by a drone to identify the location of a user so an image may be captured.

Information available from a social network (social graph information 134), for example through Facebook Connect. This information may be used to extract Subject Face 171 information for one or more friends of a user, which in turn may be provided to the drone for use in facial recognition and user tracking.

FIG. 1G illustrates an exemplary data structure for representing remuneration information policy information 180. The table below describes a plurality of remuneration policies. The user 10 may agree to one or more of the policies (i.e. they may be used alone or in any combination).

TABLE-US-00005 REMUNERATION POLICY INFORMATION DESCRIPTION Contribute User 10 agrees to contribute one or images to a Images 181 sharing pool associated with the images captured by the drone device 40 in exchange for access to the image(s) captured by the drone device 40. Watch Ads 182 User 10 agrees to watch ads in exchange for access to the image(s) captured by the drone device 40. Purchase Prints 183 User 10 agrees to use fulfillment services identified by drone device 40 in exchange for access to the image(s) captured by the drone device 40. Purchase Images 184 User 10 agrees to purchase one or more images captured by drone device 40 in exchange for access to the image(s) captured by the drone device 40. Join Network 185 User 10 creates an account, providing contact information, and agrees to receive notifications (e-mails for example) in exchange for access to the image(s) captured by the drone device 40. Tracking Information User 10 agrees to let server device 60 access Permissions 186 geographic location tracking information 176 of the user in exchange for access to the image(s) captured by the drone device 40. Social Graph User 10 agrees to let server device 60 access Information social graph information 134 of the user in Permissions 187 exchange for access to the image(s) captured by the drone device 40.

FIG. 1H illustrates an exemplary data structure for representing image metadata 190.

TABLE-US-00006 IMAGE METADATA DESCRIPTION Time 191 Time indicates the time at which the image was captured and is stored by the capture device. Date 192 Date indicates the date at which the image was captured and is stored by the capture device. Geographic Geographical location indicates the location at which the Location image was captured and is stored by the capture device. 193 The geographical location is typically stored as GPS coordinates, but may be stored in other formats such as What3Words and the like. The geographical location may be determined from cell tower triangulation, WIFI network triangulation, GPS, or any combination thereof. Camera The make of the camera that captured the image. make 194 Camera The model of the camera that captured the image. model 195 Capture The camera settings at the time the image was capture. settings 196 camera settings typically include aperture, exposure, shutter speed, focus mode, etc. Altitude 197 The height at which the camera was when the image was capture in relation to a base level. Sea level is typically used as a base level. Compass The direction at which the camera was pointed when the direction image was captured. Typically in units of milliradians. 198

While the system described herein has been described in terms of capturing still images, it will be apparent to one of ordinary skill that most of the principals described herein would apply to audio and video as well as thus should be considered within the scope of this application. In another aspect of the following disclosure, the principals are applied to key frames extracted from a video capture steam.

FIG. 2A illustrates a network diagram 200 for operating the system of FIG. 1A and describes the transfer of image data over the radio link 16. Control and status messages are sent 202 between the user device 20 and server device 60 over the network 15. Control and status messages are sent 204 between the server device 60 and drone operator device 80. Control and status messages are sent 206 between the drone operator device 80 and drone device 40 over the radio link 16. Upon capture of images at the drone device 40, image data is sent from the drone device 40. In some embodiments, reduced resolution images are sent first to provide more timely feedback. For example, reduced resolution images are sent 208 from the drone device 40 to the drone operator device 80 over the radio link 16. The reduced resolution images are then sent 210 from the drone operator device 80 to the server device 60 over the network 15. The reduced resolution images are then sent 212 from the server device 60 to the user device over the network 15. In some embodiments, based on input received from the user 10, the user device 20 may request 214 full resolution images from the server device 60 over the network 15. The server device 60 may then relay 216 the request to the drone operator device 80 over the network 15. The drone operator device 80 may then relay 218 the request to the drone device 40 over the radio link 16. In response, the drone device 40 may send 220 full resolution images to the drone operator device 80 over the radio link 16. The full resolution images are then sent 222 from the drone operator device 80 over the network 15 to the server device 60 and forwarded 224 to the user device 20 over the network 15. In some embodiments, only full resolution images are sent. In some embodiments, images, either reduced or full resolution are sent without being requested. In some embodiments, images, either reduced or full resolution are sent only when requested. In some embodiments, information identifying an address where the images may be retrieved is sent instead of the images, and the recipient of the address may retrieve the image data at the indicated address. Images may be transferred individually or in batches.

FIG. 2B illustrates a network diagram 230 for operating the system of FIG. 1A and describes the transfer of image data over the network 15. In the embodiment described in FIG. 2B, the server device 60 is coupled to the drone device 40 directly over the network 15, and indirectly through the drone operator device 80 over the radio link 16. As a result, In FIG. 1A, the drone device may provide image information over the radio link 16 as in FIG. 2A, but also over the network as described in FIG. 2B. Control and status messages are sent 232 between the user device 20 and server device 60 over the network 15. Control and status messages are sent 234 between the server device 60 and drone operator device 80 over the network 15. Control and status messages are sent 236 between the drone operator device 80 and drone device 40 over the radio link 16. Control and status messages are sent 238 between the server device 60 and drone device 40 over the network 15. In some embodiments, reduced resolution images are sent first to provide more timely feedback. For example, reduced resolution images are sent 240 from the drone device 40 to the server device 60 over the network 15. The reduced resolution images are then sent 242 from the server device 60 to the user device 20 over the network 15. In some embodiments, based on input received from the user 10, the user device 20 may request 244 full resolution images from the server device 60 over the network 15. The server device 60 may then relay 246 the request the request to the drone device 40 over the network 15. In response, the drone device 40 may send 248 full resolution images to the server device 60 over the network 15, which are then forwarded 250 to the user device 20 over the network 15. In some embodiments, only full resolution images are sent. In some embodiments, images, either reduced or full resolution are sent without being requested. In some embodiments, images, either reduced or full resolution are sent only when requested. In some embodiments, information identifying an address where the images may be retrieved is sent instead of the images, and the recipient of the address may retrieve the image data at the indicated address. Images may be transferred individually or in batches.

FIG. 2C illustrates a network diagram 260 for operating the system of FIG. 1A and describes the transfer of image data employing a removal storage medium. According to FIG. 2C, image data is transferred 268 from the drone device 40 by the drone operator 12 to the drone operator device 80 using the removable storage medium 18. Reduced resolution may be sent 270 from the drone operator device 80 to the server device 60 over the network 15 and relayed 272 to the user device 20 by the server device 60 over the network 15. In some embodiments, based on input received from the user 10, the user device 20 may request 274 full resolution images from the server device 60 over the network 15, and the request may be relayed 276 from the server device 60 over the network 15 to the drone operator device 80. In response, full resolution images may be sent 278 to the server device 60 over the network 15 and relayed 280 from the server device 60 to the user device 20. In some embodiments, only full resolution images are sent. In some embodiments, images, either reduced or full resolution are sent without being requested. In some embodiments, images, either reduced or full resolution are sent only when requested. In some embodiments, information identifying an address where the images may be retrieved is sent instead of the images, and the recipient of the address may retrieve the image data at the indicated address. Images may be transferred individually or in batches.

FIG. 3 illustrates a network diagram 300 for a scenario where a drone device is instructed to capture images found in an existing sharing pool. Referring now to FIG. 3, the user device 20 captures a first plurality of images 301 which are sent 302 to the server device 60. In some embodiments, the server device 60 will form 304 a sharing pool around the first plurality of images, or if a sharing pool already exists, it will add the first plurality of images to the existing sharing pool. The server device 60 will analyze 306 the first plurality of images to determine factors-of-interest. Factors-of-interest may include subject faces, geographical locations, subject tracking information, user tracking information, etc. These factors-of-interest are sent 308 to a drone device 40. In some embodiments, the drone device 40 operates independently to capture 309 images based on the factors-of-interest. In some embodiments, the factors-of-interest are made available to a drone operator device 80, and the drone operator device 80 guides the drone device 40 through the capture process. In some embodiments, it is a combination of the drone device 40 operating independently at times, and at other times the drone operator device 80 is controlling or guiding the drone device 40. The second plurality of images are sent 310 to the server device 40. One or more of the second plurality of images are sent 312 to the user device 20.

As used herein, the term "subject" refers to a person who may or may not be a user 10 of the system. A subject is typically a person who appears in an image or has been identified for capture. As used herein, the term "subject face" refers to the face of the subject.

In some embodiments, the factors-of-interest will be sent directly to the drone device 40 and not through the drone operator device 80. The drone operator device 80 will operate the drone device 40 to capture images in accordance with the information and directions received from the server device 60.

In some embodiments, a drone device 40 will take an image comprising factors-of-interest and send it to the server device 60. Based on the received image, the server device 60 will isolate the factors-of-interest and send them to the drone device 40. In some embodiments, the drone device 40 will determine the factors-of-interest and send them to the server device 60. In some embodiments, the drone device 40 will enable the user 12 of the drone device 40 to input the factors-of-interest explicitly, and those will be sent to the server device 60.

FIG. 4 illustrates a network diagram 400 for a scenario where a drone device 40 is instructed to capture images based on identifying targets from a social network. Referring now to FIG. 4, the user device 20 signs 402 into the server device 60 which also operates as or in conjunction with a social networking service. Based on inputs received from the user 10, the user device 20 also grants permission to the server device 60 to retrieve 404 information from the social network server 130 such as social graph information 134. The server device 60 analyzes 406 the social graph information 134 to determine capture target subjects. The target subject information is sent 408 to the drone device 40. The drone captures 410 images based on the target subject information. The captured images are sent 412 to the server device 60. Using the social graph information 134, the images are analyzed to verify 414 that they contain target subjects. The server device 60 analyzes 416 the social graph to match up addresses for the subjects with images in which they appear. The images are distributed 418 to the user 10 and or the user's friends 10[2:N]. In some embodiments, ones of the plurality of images 420 are provided to the user 10. In some embodiments, ones of the plurality of images are sent to ones of the plurality of the user's friends. In some embodiments, ones of the plurality of images are sent to subjects that appear in at least one of the captured images. In some embodiments, subjects are only sent the images in which they appear.

In some embodiments, user devices 20 provide factors-of-interest information and remuneration policy information. The factors-of-interest may include subject faces, geographic locations, timing information, subject identifiers, etc. The remuneration policy information is used to identify the remuneration the user 10 is prepared to offer in exchange for the drone device 40 capturing images meeting their factors-of-interest information. Based on this information, the server device 40 prioritizes the images that will be captured by the drone device 40. For example, a user may provide a subject face as factors-of-interest information and a price which they are willing to pay for images meeting the factors-of-interest information.

FIG. 5A illustrates a network diagram 500 for a scenario where a drone device is instructed to capture images based on remuneration policy information. In other words, the drone device 40, under the guidance of the server device 60, attempts to maximize a financial outcome of operating the drone by capturing a group of images that are likely to generate the most financial gain within an event. Referring now to FIG. 5, a first user device 20-1 sends 502 first factor-of-interest information and sends 504 first remuneration policy information to the server device 60. A second user device 20-2 sends 506 second factor-of-interest information and sends 508 second remuneration policy information to the server device 60. The server device 60 determines 510 capture priorities based on the first remuneration policy and the second remuneration policy. Aggregate factor-of-interest information and corresponding capture priorities are sent to the drone device 40. The drone device 40 captures 514 images based on the aggregate factor-of-interest information and corresponding capture priorities. The captured images are sent 516 to the server device 60. First ones of the captured images are sent 518 to the first user device 20-1, and first remuneration is collected 520 at the server device 60. Second ones of the captured images are sent 522 to the second user device 20-1, and second remuneration is collected 524 at the server device 60.

FIG. 5B illustrates an exemplary computation for determining capture priority scores according to one aspect of the present disclosure. Each user account 530-1:4 is given points to allocate among the subject faces 171-1:6 based on the number of remuneration policies 181-187 the user account has agreed to. The points 680 allocated to user account A is 4, user account B is 2, user account C is 1 and user account D is 3 as computed in FIG. 6B 670:1-4. The user of the user account 530-1:4 is then enabled to spread those points 680 across 6 subject faces 171-1:6. A score is determined 560-1:6 for each subject face by summing the number of points 680 each user assigned to the subject face 171-1:6. The result of the computation shows that subject face B has the highest score of 4, followed by subject face C with a score of 3, subject face A is 2, subject face E is 1, followed sub subject faces D & F with a score of 0. The capture priority for a subject face 171-1:6 is then determined by sorting the subject faces 171-1:6 according to their scores 560-1:6. The drone device is thus instructed to prioritize capturing the subject faces 171-1:6 based on their rank in the sort.

In some embodiments, the server device 60 may obtain event information for several events. Each event may have multiple participants, each with their own factors-of-interest information and remuneration policies. The server device 60 may determine for each event, an opportunity score, where the opportunity score represents an estimated amount of remuneration for a drone device 40 covering the event. In some embodiments, the server device 60 may assign a drone device 40 to an event based on the opportunity score. In some embodiments, the opportunity scores are sent to the drones, and the drones are allowed to determine the event they would like to cover.

FIG. 6A illustrates a network diagram 600 for a scenario where a drone device 40 is assigned to service an event of a plurality of events. The drone device 40 is assigned to the event based on an event opportunity score. Referring now to FIG. 6A, the server device 60 obtains event information for a plurality of events 602. The server device 60 determines, based on the event information for a plurality of event opportunities 604, a plurality of corresponding opportunity scores. The server device 60 designates 606, based on the event opportunity scores, a designated event, and assigns 608 the designated drone device 40. The drone device 40 captures 610 images. The captured images are sent 612 to the server device 60. First ones of the captured images are sent 614 to the first user device 20-1, and first remuneration is collected 616 at the server device 60. Second ones of the captured images are sent 618 to the second user device 20-2, and second remuneration is collected 620 at the server device 60. In some embodiments, the images are provided as available or requested, and remuneration is batched at a later date.

FIG. 6B illustrates an exemplary computation for determining event opportunity scores. Three events are shown 630-1:3. Event One 630-1 has four participants (User Account A, B, C, D), Event Two 630-2 has two participants (User Account A,B), and Event Three 630-3 has three participants (User Account A,B,C). The seven factors shown in FIG. 6B are shown along the top 181:187. Each of the seven factors is assigned a weight 650. The sub totals 660-X are determined by taking the product of the weights 650 and the occurrence counts for each participant. The opportunity scores 640-X are determined by taking the sum of the sub totals 660. In some embodiments, the event with the highest opportunity score is designated 606.

FIG. 7 illustrates a network diagram 700 for a scenario where the drone device captures one or more images, and the server device 60 analyzes the images to determine one or more users appearing in the images. The server device 60 then invites the one or more users by matching the subject faces to the users of a social network using facial recognition and location history, and using address information from the social network to invite the users to join a sharing pool. Referring now to FIG. 7, the server device 60 first registers 701 the presence of the user device 20 associated with the user 10. The drone device 40 captures a first plurality of images 702 which are sent 704 to the server device 60. In some embodiments, the server device 60 will form 704 a sharing pool around the first plurality of images, or if a sharing pool already exists, it will add the first plurality of images to the existing sharing pool. The server device 60 will analyze 706 the first plurality of images to determine factors-of-interest. Factors-of-interest may include subject faces, geographical locations, subject tracking information, user tracking information, etc. Based on the factors-of-interest, addressing information based on ones of the factors-of-interest is determined 708. An invitation is sent 710 using the addressing information. The invitation is presented 712 at the user device 20. If the invitation is accepted, an acceptance is sent 714 to the server device 60. Ones of the plurality of images captured by the drone are sent to the user device 20.

In some embodiments, a user of a user device 20 may sign into the drone service using their social networking credentials. By doing so, this enables the drone service to provide the drone device 40 and/or drone operator device 80 and drone operator 12 with information from the users 10 social graph. This social graph information 134 may comprise information such as a subject face image of the user 10 and subject face images of the friends of that user. The social graph information 134 may also include geographical location information for the user and the friends of the user (including the current location which may be periodically updated). Using the social graph information 134 for the user and their friends, the drone may be able to better locate and take images of the users as they move around.

FIG. 8 illustrates a network diagram 800 for a scenario where one or more drones are registered with the server device, and the server device matching and assignment of a drone device of the one or more drone devices to the requesting mobile device. Referring now to FIG. 8, a drone device 40-1 registers 802 with the server device 60 and sends 803 capability information. Other drones 40-[2-N] register 804 with the server device 60 and provide 805 other capability information for each of the other respective drones 40-[2-N]. The server device 60 receives 806 a request for a drone device 40, the request including a start time, start location, and capture capabilities. The server device 60 determines 808 a drone based on the request information and assigns 810 the drone to the user device 20. Notification of assignment is sent 812 to the user device 20. Once the start time is reached 814 and the user device is in control of the drone device 40, another notification is sent to the user device 816. The user device 20 sends 818 control commands to the drone and receives status information. Guided by the user device 20, the drone device 40 captures 820 images and sends 822 them to the user device 20.

FIG. 9 is an aerial view 900 illustrating the use of drone imaging at a sporting event, in this case a soccer game. A drone device 40 is operated by a drone operator 12. A user device 20 is operated by a user 10 who may also be a pool participant and/or an initiator. A subject 14 may be any person attending the event, including the operator 12 and the user 10.

FIG. 10 is an aerial view 1000 illustrating the use of drone imaging at a vacation event, in this case a cruise. Multiple drone devices 40-1 and 40-2 are operated by corresponding drone operators 12-1 and 12-2. User device 20-1 and 20-2 (not shown) are operated by users 10-1 and 10-2 who may also be pool participants and/or initiator. Subjects 14-1 and 14-2 may be any person attending the event, including the operators 12-1 and 12-2 and users 10-1 and 10-2.

FIG. 11 is an aerial view 1100 illustrating the use of drone imaging at a public event, in this case a parade. Multiple drone devices 40-1 and 40-2 are operated by corresponding drone operators 12-1 and 12-2. User device 20-1 and 20-2 (not shown) are operated by users 10-1 and 10-2 who may also be pool participants and/or initiator. Subjects 14-1 and 14-2 may be any person attending the event, including the operators 12-1 and 12-2 and users 10-1 and 10-2.

FIG. 12 is an aerial view 1200 illustrating the use of drone imaging at a public structure viewing event, in this case a visit to the Eiffel Tower. Multiple drone devices 40-1 and 40-2 are operated by corresponding drone operators 12-1 and 12-2. User device 20-1 and 20-2 (not shown) are operated by users 10-1 and 10-2 who may also be pool participants and/or initiator. Subjects 14-1 and 14-2 may be any person attending the event, including the operators 12-1 and 12-2 and users 10-1 and 10-2.

FIG. 13 is an aerial view 1300 illustrating the use of drone imaging at an outdoor structure viewing event, in this case the Golden Gate Bridge. A drone devices 40-1 is operated by a corresponding drone operator 12-1. User device 20-1 (not shown) is operated by user 10-1 who may also be pool participants and/or initiator. Subject 14-1 may be any person attending the event, including the operator 12-1 and users 10-1.

FIG. 14 is a side view of an exemplary drone device 40. The drone device 40 is comprised of a control system 41, propulsion system 1402 and an imaging subsystem 1404. An example of a moderately priced commercially available consumer drone with image capture features is the DJI Phantom Drone line. The DJI Phantom 3 Standard has a flight time of 25 minutes, captures video at up to 2.7K at 30 fps. The remote communicates with the drone at ranges up to 1000 meters over a 2.7 KHz radio link 16. The imaging subsystem 1404 may be a built-in component, an external component such as a GoPro camera, or some combination thereof.

FIG. 15 is a block diagram of a user device 20 according to one embodiment of the present disclosure. As illustrated, the user device 20 includes a controller 1504 connected to memory 1506, one or more communications interfaces 1508, one or more user interface components 1510, one or more storage components 1512, and a location component 1514 by a bus 1502 or similar mechanism. The controller 1504 is a microprocessor, digital ASIC, FPGA, or the like. In general, the user device 20 includes a control system 21 having associated memory 1506. In this embodiment, the controller 1504 is a microprocessor, and the user interface (UI) module 22, communications module 23, imaging module 24, and configuration module 25 are implemented in software and stored in the memory 1506 for execution by the controller 1504. However, the present disclosure is not limited thereto. The aforementioned functions and module may be implemented in software, hardware, or a combination thereof. The user device 20 also includes a communication interface 1508 enabling the user device 20 to connect to the network 15. The one or more user interface components 1510 include, for example, a touchscreen, a display, one or more user input components (e.g., a keypad), a speaker, or the like, or any combination thereof. The storage component(s) 1512 is a non-volatile memory. In this embodiment, the location component 1514 is a hardware component, such as a GPS receiver. However, the present invention is not limited thereto.

FIG. 16 is a block diagram of a drone device 40 according to one embodiment of the present disclosure. As illustrated, the drone device 40 includes a controller 1604 connected to memory 1606, one or more communications interfaces 1608, one or more user interface components 1610, location component 1612, one or more storage components 1614, guidance sensors 1616 by a bus 1602 or similar mechanism. The controller 1604 is a microprocessor, digital ASIC, FPGA, or the like. In general, the drone device 40 includes a control system 41 having associated memory 1606. In this embodiment, the controller 1604 is a microprocessor, and the guidance module 42, communications module 43, power management 44, and capture module 45, storage module 46, and a synchronization module 47 are implemented in software and stored in the memory 1606 for execution by the controller 1604. However, the present disclosure is not limited thereto. The aforementioned functions and module may be implemented in software, hardware, or a combination thereof. The user device 20 also includes a communication interface 1608 enabling the drone device 40 to connect to the network 15 and to the drone operator device 80 over the radio link 16. The one or more user interface components 1610 include, for example, a touchscreen, a display, one or more user input components (e.g., a keypad), a speaker, or the like, or any combination thereof. In this embodiment, the location component 1612 is a hardware component, such as a GPS receiver. The storage component(s) 1614 is a non-volatile memory of any number of forms. The guidance sensors 1616 may be one or more of accelerometers, engine intake flow sensors, magnetic sensors, tilt sensors, current sensors, and inertial measurement units.

Referring now to the various guidance sensors 1616, accelerometers are used to determine position and orientation of the drone in flight. Like your Nintendo Wii controller or your iPhone screen position, these small silicon-based sensors play a key role in maintaining flight control. MEMS accelerometers sense movement in several ways. One type of technology senses the micro movement of very small structures embedded small integrated circuit. The movement of these small `diving boards` change the amounts of electrical current moving through the structure, indicating change of position relative to gravity. Another technology used in accelerometers is thermal sensing which offers several distinct advantages. It does not have moving parts, but instead senses changes in the movement of gas molecules passing over a small integrated circuit. Because of the sensitivity of these sensors, they play a role in stabilizing on-board cameras, which are vital for applications like filmmaking. By controlling up and down movement, as well as removing jitter and vibration, filmmakers are able to capture extremely smooth looking video. Additionally, because these sensors are more immune to vibrations than other technologies, thermal MEMS sensors are perfect in drone applications to minimize problems from the increased vibration generated by the movement of rotating propulsion fans/propellers.

Combined with GPS, inertial measurement units (IMUs) are critical for maintaining direction and flight paths. As drones become more autonomous, these are essential to maintain adherence to flight rules and air traffic control. IMUs units use multi-axis magnetometers that are, in essence, small, accurate compasses. These sense changes in direction and feed data into a central processor, which ultimately indicates direction, orientation, and speed.

Tilt sensors, combined with gyros and accelerometers provide input to the flight control system in order to maintain level flight. This is extremely important for applications where stability is paramount, from surveillance to delivery of fragile goods. These types of sensors combine accelerometers with gyroscopes, allowing the detection of small variations of movement. It is the gyroscope compensation that allows these tilt sensors to be used in moving applications like motor vehicles or drones.

In drones, power consumption and use are important, particularly those that are battery powered. Current sensors can be used to monitor and optimize power drain, safe charging of internal batteries, and detect fault conditions with motors or other areas of the system. Current sensors work by measuring electrical current (bi-directional) and ideally provide electrical isolation to reduce power loss and eliminate opportunity for electrical shock or damage to the user or systems.

Sensors with fast response time and high accuracy optimize the battery life and performance of drones.

In drones, electronic compasses provide critical directional information to inertial navigation and guidance systems. Anisotropic magnetoresistive (AMR) permalloy technology sensors, which have superior accuracy and response time characteristics while consuming significantly less power than alternative technologies, are well-suited to drone applications. Turnkey solutions provide drone manufacturers with quality data sensing in a very rugged and compact package.

Engine intake flow sensors can be used to effectively monitor air flow into small gas engines used to power some drone varieties. These help the engine CPU determine the proper fuel-to-air ratio at specified engine speed, which results in improved power and efficiency, and reduced emissions. Many gas engine mass flow sensors employ calorimetric principal utilizing a heated element and at least one temperature sensor to quantify mass flow. MEMS thermal mass air flow sensors also utilize calorimetric principal but in a micro scale, making it highly suitable for applications where reduced weight is critical.

FIG. 17 is a block diagram of a server device 60 according to an embodiment of the present disclosure. As illustrated, server device 60 includes a controller 1704 connected to a memory 1706, one or more secondary storage components 1712, and one or more communications interfaces 1708 by a bus 1702 or similar mechanism. The controller 1704 is a microprocessor, digital Application Specific Integrated Circuit ASIC, Field Programmable Gate Array FPGA, or the like. In general, the server device 60 includes a control system 61 having associated memory 1706. In this embodiment, the controller 1704 is a microprocessor, and the event management module 62, user management module 63, device management module 64, drone management module 65, image analysis engine 66, and invitation management module 67 are implemented in software and stored in the memory 1706 for execution by the controller 1704. However, the present disclosure is not limited thereto. The aforementioned module may be implemented in software, hardware, or a combination thereof. Further, the event registry 100, user registry 102, devices registry 104 and drones registry 106 may be stored in the one or more secondary storage components 1712. The secondary storage components 1712 are digital data storage components such as, for example, one or more hard disk drives. The server device 60 also includes a communication interface 1708 enabling the server device 60 to connect to the network 15.

Those skilled in the art will recognize improvements and modifications to the embodiments of the present disclosure. All such improvements and modifications are considered within the scope of the concepts disclosed herein and the claims that follow.

* * * * *

Патент США №

Автор(ы)

Дата выдачи

Determining capture instructions for drone photography based on information received from a social network

РЕФЕРАТ

Авторы:

Патентообладатель:

ID семейства патентов

Номер заявки:

Дата регистрации:

Отсылочные патентные документы США

Класс патентной классификации США:

Класс совместной патентной классификации:

Класс международной патентной классификации (МПК):

Область поиска: