Lex Fridman

Massachusetts Institute of Technology

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Bryan Reimer

New England University Transportation Center

Bruce Mehler

New England University Transportation Center

Jack Terwilliger

University of California, San Diego

Joonbum Lee

University of Wisconsin–Madison

Michael Glazer

Massachusetts Institute of Technology

Benedikt Jenik

Massachusetts Institute of Technology

Bobbie Seppelt

Ford Motor Company (United States)

Li Ding

Beijing Institute of Technology

V. A. Perlin

СФ

С. Ф. Гребенников

Saint-Petersburg State University of Technology and Design

Cooperative Institutions

Massachusetts Institute of Technology

New England University Transportation Center

Research Institute of Synthetic Fiber with the Pilot Plant

Moscow Institute of Thermal Technology

Russian Academy of Sciences

IIT@MIT

Technical University of Munich

University of California, Berkeley

Stanford University

Drexel University

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Driver frustration detection from audio and video in the wild

International Joint Conference on Artificial Intelligence (2016)

Irman Abdić Lex Fridman Daniel McDuff Erik Marchi Bryan Reimer

We present a method for detecting driver frustration from both video and audio streams captured during the driver's interaction with an in-vehicle voice-based navigation system. The video is of the driver's face when the machine is speaking, and the audio is of the driver's voice when he or she is speaking. We analyze a dataset of 20 drivers that contains 596 audio epochs (audio clips, with duration from 1 sec to 15 sec) and 615 video epochs (video clips, with duration from 1 sec to 45 sec). The dataset is balanced across 2 age groups, 2 vehicle systems, and both genders. The model was subject-independently trained and tested using 4- fold cross-validation. We achieve an accuracy of 77.4% for detecting frustration from a single audio epoch and 81.2% for detecting frustration from a single video epoch. We then treat the video and audio epochs as a sequence of interactions and use decision fusion to characterize the trade-off between decision time and classification accuracy, which improved the prediction accuracy to 88.5% after 9 epochs.

CLIPS

Source

Cite

Citations (32)

Object as Distribution

arXiv (Cornell University) (2019)

Li Ding Lex Fridman

Object detection is a critical part of visual scene understanding. The representation of the object in the detection task has important implications on the efficiency and feasibility of annotation, robustness to occlusion, pose, lighting, and other visual sources of semantic uncertainty, and effectiveness in real-world applications (e.g., autonomous driving). Popular object representations include 2D and 3D bounding boxes, polygons, splines, pixels, and voxels. Each have their strengths and weakness. In this work, we propose a new representation of objects based on the bivariate normal distribution. This distribution-based representation has the benefit of robust detection of highly-overlapping objects and the potential for improved downstream tracking and instance segmentation tasks due to the statistical representation of object edges. We provide qualitative evaluation of this representation for the object detection task and quantitative evaluation of its use in a baseline algorithm for the instance segmentation task.

Robustness

Representation

Bounding overwatch

Source

Cite

Citations (2)

Driver Gaze Estimation Without Using Eye Movement

arXiv (Cornell University) (2015)

Lex Fridman Philipp Langhans Joonbum Lee Bryan Reimer

Abstract—Automated estimation of the allocation of adriver’s visual attention may be a critical component offuture Advanced Driver Assistance Systems. In theory,vision-based tracking of the eye can provide a good estimateof gaze location. In practice, eye tracking from videois challenging because of sunglasses, eyeglass reﬂections,lighting conditions, occlusions, motion blur, and otherfactors. Estimation of head pose, on the other hand, isrobust to many of these effects, but cannot provide as ﬁne-grained of a resolution in localizing the gaze. However,for the purpose of keeping the driver safe, it is sufﬁcientto partition gaze into regions. In this effort, we proposea system that extracts facial features and classiﬁes theirspatial conﬁguration into six regions in real-time. Ourproposed method achieves an average accuracy of 91.4%at an average decision rate of 11 Hz on a dataset of 50drivers from an on-road study.Index Terms—Head pose estimation, gaze tracking, driverdistraction, driver assistance systems, on-road study.

Source

Cite

Citations (14)

Active Authentication on Mobile Devices via Stylometry, Application Usage, Web Browsing, and GPS Location

IEEE Systems Journal (2016)

Lex Fridman Steven Weber Rachel Greenstadt Moshe Kam

Active authentication is the problem of continuously verifying the identity of a person based on behavioral aspects of their interaction with a computing device. In this study, we collect and analyze behavioral biometrics data from 200subjects, each using their personal Android mobile device for a period of at least 30 days. This dataset is novel in the context of active authentication due to its size, duration, number of modalities, and absence of restrictions on tracked activity. The geographical colocation of the subjects in the study is representative of a large closed-world environment such as an organization where the unauthorized user of a device is likely to be an insider threat: coming from within the organization. We consider four biometric modalities: (1) text entered via soft keyboard, (2) applications used, (3) websites visited, and (4) physical location of the device as determined from GPS (when outdoors) or WiFi (when indoors). We implement and test a classifier for each modality and organize the classifiers as a parallel binary decision fusion architecture. We are able to characterize the performance of the system with respect to intruder detection time and to quantify the contribution of each modality to the overall performance.

Modality (human–computer interaction)

Spoofing attack

Modalities

10.1109/jsyst.2015.2472579

Cite

Citations (170)

Pore structure and adsorptive properties of activated fibrous carbon materials

Fibre Chemistry (1982)

S. F. Brebennikov Yu. I. Pakhomov L. V. Novinyuk Lex Fridman O. A. Portnoi

Carbon fibers

10.1007/bf00548052

Cite

Citations (0)

On the joint impact of bias and power control on downlink spectral efficiency in cellular networks

arXiv (Cornell University) (2015)

Lex Fridman Jeffrey Wildman Steven Weber

Cell biasing and downlink transmit power are two controls that may be used to improve the spectral efficiency of cellular networks. With cell biasing, each mobile user associates with the base station offering, say, the highest biased signal to interference plus noise ratio. Biasing affects the cell association decisions of mobile users, but not the received instantaneous downlink transmission rates. Adjusting the collection of downlink transmission powers can likewise affect the cell associations, but in contrast with biasing, it also directly affects the instantaneous rates. This paper investigates the joint use of both cell biasing and transmission power control and their (individual and joint) effects on the statistical properties of the collection of per-user spectral efficiencies. Our analytical results and numerical investigations demonstrate in some cases a significant performance improvement in the Pareto efficient frontiers of both a mean-variance and throughput-fairness tradeoff from using both bias and power controls over using either control alone.

Biasing

Spectral Efficiency

Transmitter power output

Source

Cite

Citations (0)

Deep Learning for Understanding the Human

Lex Fridman

We will explore how deep learning approaches can be used for perceiving and interpreting the state and behavior of human beings in images, video, audio, and text data. The course will cover how convolutional, recurrent and generative neural networks can be used for applications of face recognition, eye tracking, cognitive load estimation, emotion recognition, natural language processing, voice-based interaction, and activity recognition. The course is open to beginners and is designed for those who are new to deep learning, but it can also benefit advanced researchers in the field looking for a practical overview of deep learning methods and their application.

10.1145/3170427.3170661

Cite

Citations (0)

Dynamics of the adsorption of vapors and gases by fibrous carbon adsorbents

Fibre Chemistry (1981)

Lex Fridman V. A. Perlin L. V. Talalai V. N. Akhmatov

Carbon fibers

10.1007/bf00549119

Cite

Citations (0)

Intrapore reaction of carbon fibers with oxygen-containing gases

Solid Fuel Chem. (Engl. Transl.); (United States) (1983)

V.I. Elchina Lex Fridman V.M. Gurdzhiyants V.D. Ivin R. M. Levit

A study has been made of the conditions of the internal reaction of carbon fiber obtained from polyacrylonitrile under various conditions of heat thermal treatment (the final temperature was varied from 880/sup 0/C to 2400/sup 0/C) with gaseous oxidizing agents: atmospheric oxygen, carbon dioxide, and steam.

Polyacrylonitrile

Oxidizing agent

Carbon fibers

Source

Cite

Citations (1)

Owl and Lizard: Patterns of Head Pose and Eye Pose in Driver Gaze Classification

arXiv (Cornell University) (2015)

Lex Fridman Joonbum Lee Bryan Reimer Trent Victor

Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (1) vehicle interfaces and (2) the design of future Advanced Driver Assistance Systems. In this paper, we estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can we classify driver gaze using head and eye pose versus just using head pose? Second, are there individual-specific gaze strategies that strongly correlate with how much gaze classification improves with the addition of eye pose information? We answer these questions by evaluating data drawn from an on-road study of 40 drivers. The main insight of the paper is conveyed through the analogy of an owl and lizard which describes the degree to which the eyes and the head move when shifting gaze. When the head moves a lot (owl), not much classification improvement is attained by estimating eye pose on top of head pose. On the other hand, when the head stays still and only the eyes move (lizard), classification accuracy increases significantly from adding in eye pose. We characterize how that accuracy varies between people, gaze strategies, and gaze regions.

Monocular

Source

Cite

Citations (1)