June 2024

This month’s newsletter features Amazon’s research at ICML 2024, CVPR 2024, and NAACL 2024, awards and recognitions in our science community, and several generative AI updates. 

Interpretable ensemble models improve product retrieval: New information retrieval models are constantly being released, but evaluating them takes time. At The Web Conference, Amazon scientists proposed adding new models to an ensemble and then using Shapley value analysis to determine whether to keep them.

Five ways Amazon is preparing for the energy demands of the future: From investing in new carbon-free energy projects to advocating for grid modernization and collaborating with key stakeholders around the world, learn how Amazon is working toward a cleaner energy future.

Automated evaluation of RAG pipelines with exam generation: Retrieval-augmented generation (RAG) is a leading way to curb "hallucination" in large language models (LLMs), and at this year’s International Conference on Machine Learning (ICML 2024), Amazon researchers will show how to leverage item response theory to automatically generate "exams" for evaluating RAG approaches.

Amazon researchers receive Best Paper Award at the Symposium on Foundations of Responsible Computing (FORC 2024) The paper, co-authored by Tiffany (Siqi) Deng, AGI applied science manager, Emily Diana, research assistant professor at the Toyota Technological Institute at Chicago, and Michael Kearns and Aaron Roth, Amazon Scholars and UPenn professors, addresses the challenge of creating a balanced dataset when sensitive-group information is unavailable at deployment time. The researchers propose using a small labeled dataset to train a proxy function that assigns sampling probabilities based on the proxy classification, without revealing significantly more about the group membership of any individual sample than can be ascertained from base rates alone.

A quick guide to Amazon’s papers at NAACL 2024: Unsurprisingly, work involving LLMs, either as a subject of inquiry themselves or as tools for other natural-language-processing applications, predominates at this year’s conference. This paper guide sorts Amazon’s papers into those that deal explicitly with LLMs and those that don’t — although in many cases, the ones that don’t present general techniques or datasets that could be used with either LLMs or more-traditional models.

A quick guide to Amazon’s papers at CVPR 2024: A plurality of the papers deal with vision-language models, while a number of others concern related topics such as visual question answering, hallucination mitigation, and retrieval-aided generation. At the same time, however, classical computer vision topics such as 3-D reconstruction, object tracking, and pose estimation remain well represented.

Amazon Research Award-funded paper receives Best Paper Award: With the support of an Amazon Research Award, a team from Imperial College London and Amazon Web Services (AWS) received an Industry Track Best Paper Award at this year’s International Conference on Software Testing, Verification and Validation (ICST 2024). Their paper presents two new tools, fuzz-d and DafnyFuzz, which improves Dafny compiler testing. The researchers found 24 critical bugs, including 9 soundness issues, surpassing XDsmith, and their testing campaign led to improvements in the Dafny language specification, addressing ambiguous or under-documented language features.

Amazon Scholar honored with IEEE Photonics Society Quantum Electronics Award: Joint Quantum Institute Fellow, NIST researcher, adjunct associate professor in the Department of Physics at UMIACS, and Amazon Scholar Alexey Gorshkov at the AWS Center for Quantum Computing, received the award for his research contributions in the areas of understanding, designing, and controlling interacting quantum systems.

Amazon Visiting Academic receives NSF Early Career Award: Amazon Visiting Academic in Amazon’s AGI organization and assistant professor of computer science at the UCLA Samueli School of Engineering, NANYUN PENG, received the award from the National Science Foundation (NSF) to support her research in AI and developing a new category of generative language models. The award is the agency’s highest honor for faculty members in the early stages of their careers, where she will receive a five-year, $586,000 grant to fund her research and teaching efforts.

Anthropic’s Claude 3.5 Sonnet model now available in Amazon Bedrock: Claude 3.5 Sonnet raises the industry bar for intelligence, outperforming other generative AI models on a wide range of evaluations, including Anthropic’s previously most intelligent model, Claude 3 Opus. Learn more about the model’s strengths and key improvements.

Amazon's new AI-powered tools help advertisers easily create engaging and vibrant images: The growing suite of generative AI tools from Amazon Ads is helping brands to quickly and easily create lifestyle images around their products, elevating the customer discovery experience. Hear from Jason (Jay) Richman, vice president of product and technology for Amazon Ads, about the new aspect ratio capability and what other features his team is planning to release this year.

How small businesses can boost productivity using generative AI: Amazon Web Services principal applied scientist and founder of Bean Path, a nonprofit education organization, Nashlie Sephus, Ph.D., shares her tips for using free generative AI tools to help grow businesses. Sephus’ videos shows small-business owners and entrepreneurs how to use AWS PartyRock, a free, generative AI app-building tool, built on Amazon Bedrock to help address some pain points.

Upcoming conferences

• SIGIR 2024, July 14 - 18

• ICML 2024, July 21 - 27

• ACL 2024, August 11 - 17

• KDD 2024, August 25 - 29

New publications

• A critical analysis of circular product attributes and limitations of product circularity assessment methods

• A product-aware query auto-completion framework for e-commerce search via retrieval-augmented generation method

• A shocking amount of the web is machine translated: Insights from multi-way parallelism

• An efficient self-learning framework for interactive spoken dialog systems

• Automated evaluation of retrieval-augmented language models with task-specific exam generation

• Balanced filtering via disclosure-controlled proxies

• BASS: Batched attention-optimized speculative sampling

• Bayesian prompt ensembles: Model uncertainty estimation for black-box large language models

• Bifurcated attention for single-context large-batch sampling

• But where are you going?! Motion is what is most important for real-world co-present mobile robots

• Can your model tell a negation from an implicature? Unravelling challenges with intent encoders

• CERET: Cost-effective extrinsic refinement for text generation

• Confidence intervals for error rates in 1:1 matching tasks: Critical statistical analysis and recommendations

• Constant directivity loudspeaker beamforming

• Efficient continual pre-training for building domain specific large language models

• Electrical manipulation of telecom color centers in silicon

• Eliciting better multilingual structured reasoning from LLMs through code

• EMC2: Efficient MCMC negative sampling for contrastive learning with global convergence

• Enhancing multimodal large language models with multi-instance visual prompt generator for visual representation enrichment

• ErgoReality: A virtual reality simulations software for ergonomic analysis of workstation design

• Explaining probabilistic models with distributional values

• Exploring ordinality in text classification: A comparative study of explicit and implicit techniques

• Extreme miscalibration and the illusion of adversarial robustness

• Factual confidence of LLMs: On reliability and robustness of current estimators

• Fine-tuned machine translation metrics struggle in unseen domains

• Finite-time convergence and sample complexity of actor-critic multi-objective reinforcement learning

• Frogs into princes: A generative model to understand the success of product descriptions

• Generative explore-exploit: Training-free optimization of generative recommender systems using LLM optimizers

• Gradual fine-tuning with graph routing for multi-source unsupervised domain adaptation

• GRAM: Generative retrieval augmented matching of data schemas in the context of data security

• GraphStorm: All-in-one graph machine learning framework for industry applications

• Hector: An efficient programming and compilation framework for implementing relational graph neural networks in GPU architectures

• II-MMR: Identifying and improving multi-modal multi-hop reasoning in visual question answering

• Impacts of misspelled queries on translation and product search

• Interpretable measures of conceptual similarity by complexity-constrained descriptive auto-encoding

• Large language models (LLMs) on tabular data: Prediction, generation, and understanding - a survey

• Large language models as recommender systems: A study of popularity bias

• Learning the target network in function space

• MADA: Meta-adaptive optimizers through hyper-gradient descent

• MAML-en-LLM: Model agnostic meta-training of LLMs for improved in-context learning

• MATTER: Memory-augmented transformer using heterogeneous knowledge sources

• Membership inference attacks on diffusion models via quantile regression

• MemoryLLM: Towards self-updatable large language models

• MinPrompt: Graph-based minimal prompt data augmentation for few-shot question answering

• Model ChangeLists: Characterizing updates to ML models

• Multi-modal retrieval for large language model based speech recognition

• Multicalibration for confidence scoring in LLMs

• Multimodal learning with online text cleaning for e-commerce product search

• MultiPICo: Multilingual perspectivist irony corpus

• Multiply-robust causal change attribution

• Near-optimal regret in linear MDPs with aggregate bandit feedback

• Online adaptive anomaly thresholding with confidence sequences

• Parametric constraints for Bayesian knowledge tracing from first principles

• PEMBOT: Pareto-ensembled multi-task boosted trees

• Prompting foundational models for omni-supervised instance segmentation

• Propagation and pitfalls: Reasoning-based assessment of knowledge editing through counterfactual tasks

• Reasoning and planning with large language models in code development (survey for KDD 2024 tutorial)

• REPOFORMER: Selective retrieval for repository-level code completion

• Robust multi-task learning with excess risks

• Self-supervised multi-object tracking with path consistency

• SemiGPC: Distribution-aware label refinement for imbalanced semi-supervised learning using gaussian processes

• Sequential editing for lifelong training of speech recognition models

• Shopping trajectory representation learning with pre-training for e-commerce customer understanding and recommendation

• SODA: An adaptive bitrate controller for consistent high-quality video streaming

• SpeechGuard: Exploring the adversarial robustness of multimodal large language models

• SWAN: SubWord Alignment Network for HMM-free word timing estimation in end-to-end automatic speech recognition

• Synthesizing conversations from unlabeled documents using automatic response segmentation

• The fine-tuning paradox: Boosting translation quality without sacrificing LLM abilities

• THRONE: An object-based hallucination benchmark for the free-form generations of large vision-language models

• Token alignment via character matching for subword completion

• Tokenization matters: Navigating data-scarce tokenization for gender inclusive language technologies

• Towards unbiased calibration using meta-regularization

• Transferring knowledge from large foundation models to small downstream models

• Understanding inter-session intentions via complex logical reasoning

• Using uncertainty quantification to characterize and improve out-of-domain learning for PDEs

• Valuing an engagement surface using a large scale dynamic causal model

• Where can I park my robot? Modeling out-of-the-way parking spots in the home using room geometry

LinkedIn | X/Twitter | Facebook | Instagram | GitHub | RSS

© 1996-2024 Amazon.com, Inc. or its affiliates | Privacy | Conditions of Use