Dynamical scaling in self-associating polymer aggregates
2
Citation
24
Reference
10
Related Paper
Abstract:
The authors present a simple model that simulates the dynamics of the long flexible self-associating polymer chains immersed in water and develop a dynamical scaling description for the cluster size distribution function for irreversible aggregation. Each chain may have one or several functional groups at each end that can associate with one another. They find that the chain length is a new scaling variable that appears explicitly in the scaling function. The validity of their scaling predictions is confirmed by extensive computer simulations.Keywords:
Chain (unit)
Cluster size
Cluster size
Deposition
Exponent
Pattern Formation
Particle (ecology)
Cite
Citations (10)
Scaling up language models has been empirically shown to improve performance on a wide range of downstream tasks. However, if we were to observe worse performance as a function of scale ("inverse scaling") on certain tasks, this would indicate that scaling can also encourage behaviors that are misaligned with human preferences. The Inverse Scaling Prize (McKenzie et al. 2022) identified eleven such inverse scaling tasks, evaluated on models of up to 280B parameters and up to 500 zettaFLOPs of training compute. This paper takes a closer look at these inverse scaling tasks. We evaluate models of up to 540B parameters, trained on five times more compute than those evaluated in the Inverse Scaling Prize. With this increased range of model sizes and training compute, only four out of the eleven tasks remain inverse scaling. Six out of the eleven tasks exhibit "U-shaped scaling", where performance decreases up to a certain size, and then increases again up to the largest model evaluated (the one remaining task displays positive scaling). In addition, we find that 1-shot examples and chain-of-thought can help mitigate undesirable scaling patterns even further. U-shaped scaling suggests that the inverse scaling trend observed in McKenzie et al. (2022) may not continue to hold for larger models, which we attribute to the presence of distractor tasks that only sufficiently large models can avoid.
Cite
Citations (1)
Plot (graphics)
Cite
Citations (9)
Improved approximations or displacements and stresses, achieved by the following types of scaling, are presented: a. scaling of the initial stiffness matrix; b. a new type of scaling of displacements; and c. mixed scaling of stiffness and displacements, where the two types of scaling are combined The geometric interpretation of the various scaling types is illustrated and methods for selecting the scaling multipliers based on geometrical considerations, mathematical criteria and the reduced basis approach, are demonstrated and compared. It is shown that high quality approximations can be achieved for very large changes in cross section and geometrical variables with a small computational effort. The results presented indicate that scaling procedures have high potential in future applications where effective reanalysis is essential.
Basis (linear algebra)
Matrix (chemical analysis)
Cite
Citations (2)
Chain (unit)
Scaling law
Cite
Citations (0)
Abstract We study the scaling of the average cluster size and percolation strength of geometrical clusters for the two-dimensional Ising model. By means of Monte Carlo simulations and a finite-size scaling analysis we discuss the appearance of corrections to scaling for different definitions of cluster sets. We find that including all percolating clusters, or excluding only clusters that percolate in one but not the other direction, leads to smaller corrections to scaling for the average cluster size as compared to the other definitions considered. The percolation strength is less sensitive to the definitions used.
Percolation (cognitive psychology)
Cluster size
Scaling law
Cite
Citations (2)
Percolation (cognitive psychology)
Widom scaling
Scaling law
Cite
Citations (12)
Finite-size scaling (FSS) is a standard technique for measuring scaling exponents in spin glasses. Here we present a critique of this approach, emphasizing the need for all length scales to be large compared to microscopic scales. In particular we show that the replacement, in FSS analyses, of the correlation length by its asymptotic scaling form can lead to apparently good scaling collapses with the wrong values of the scaling exponents.
Widom scaling
Scaling law
Dynamic scaling
Cite
Citations (1)
Cite
Citations (1)
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data. In this work, we introduce NeQA, a dataset consisting of questions with negation in which language models do not exhibit straightforward positive scaling. We show that this task can exhibit inverse scaling, U-shaped scaling, or positive scaling, and the three scaling trends shift in this order as we use more powerful prompting methods or model families. We hypothesize that solving NeQA depends on two subtasks: question answering (task 1) and negation understanding (task 2). We find that task 1 has linear scaling, while task 2 has sigmoid-shaped scaling with an emergent transition point, and composing these two scaling trends yields the final scaling trend of NeQA. Our work reveals and provides a way to analyze the complex scaling trends of language models.
Negation
Cite
Citations (0)