Defined Terms and Documents       

What can the Adult Literacy and Lifeskills Survey (ALLS) tell us? - What’s ALL the story?

Dave Tout, ACER   tout@acer.edu.au

􀂃 Reports available free from: http://www.abs.gov.au/

􀂃 Data sets available to researchers

􀂃 ALLS: document no: 4228.0

􀂃 Health literacy: 4233.0

􀂃 Its predecessor, the International Adult Literacy Survey (IALS) was run in Australia 10 years ago

􀂃 Data collection for the survey was undertaken by ABS in late 2006 into early 2007

􀂃 Almost 9000 adults surveyed – aged 15-74 years – representative of total Australian population excluding remote indigenous adults

􀂃 Survey is an international survey developed by Statistics Canada and the United States' Educational Testing Service coordinated with the OECD.

ALLS in Australia measured:

􀂃 Prose Literacy - the knowledge and skills needed to understand and use information from text including editorials, news stories, poems and fiction (in IALS)

􀂃 Document Literacy - the knowledge and skills required to locate and use information contained in various formats including job applications, payroll forms, transportation schedules, maps, tables and graphics (in IALS)

􀂃 Numeracy - the knowledge and skills required to effectively manage and respond to the mathematical demands of diverse situations (new)

􀂃 Health literacy - The knowledge and skills required to understand and use information relating to health issues such as drugs and alcohol, disease prevention and treatment, safety and accident prevention, first aid, emergencies, and staying healthy (new)

􀂃 Problem Solving - the knowledge and skills required to identify a problem, search for relevant information and integrate it into a coherent problem representation, evaluating the problem situation with respect to given goals and criteria, devising a plan and monitoring its execution (new)


Why numeracy and not QL?

􀂃 The tasks were, as much as is possible in a large scale testing situation, based on adult contexts and ‘real-life’ scenarios and texts, and were open ended.

􀂃 The items are based upon simulated texts such as advertisements, newspaper articles, instructions, maps, diagrams and plans, photos, etc.

􀂃 Items can be adapted to meet national requirements re language, terminology, units, etc.

􀂃 A ruler and calculator are provided to respondents for use in the numeracy items

􀂄 There is a screening process

􀂄 The Core Task Booklet consists of six prose, document and numeracy tasks to determine the respondent's ability to undertake further skills assessment.

􀂄 Each respondent who correctly answers three questions from the Core Task Booklet were asked to complete a Main Task Booklet

􀂄 All respondents complete a comprehensive Background Questionnaire

BQ includes almost 300 questions about:

􀂃 Demographics

􀂃 Education

􀂃 Language

􀂃 Parental Information

􀂃 Labour Force

􀂃 Literacy and Numeracy Practices at work

􀂃 Literacy and Numeracy Practices generally

􀂃 Participation in Education and Learning

􀂃 Social Capital and Well Being

􀂃 Use of Technologies

􀂃 Income

Literacy and Numeracy Practices at work:

􀂃 How often reads letters, memos or emails

􀂃 How often reads or uses reports, articles, magazines or journals

􀂃 How often reads or uses manuals or reference books including catalogues

􀂃 How often reads or uses diagrams or plans

􀂃 How often reads directions or instructions

􀂃 How often reads or uses bills, invoices, spreadsheets or budget tables

􀂃 How often writes or fills in letters, memos or emails

􀂃 How often writes or fills in reports, articles, magazines or journals

􀂃 How often writes or fills in manuals or reference books including catalogues

􀂃 How often writes or fills in directions or instructions

􀂃 As in IALS, the literacy, numeracy and problem solving ability is expressed as a score on a scale ranging from 0-500 points. The score is the point at which a person has an 80% chance of successfully performing tasks at that level. (Cf PISA – 60%)

􀂃 The scale is grouped into five levels. Level 3 is considered the level adults require to cope with the demands of everyday life and work.

􀂃 The 5 levels do not directly correlate to the 5 levels of the Australian National Reporting System (NRS) or its revision, the Australian Core Skills Framework (ACSF).

These take into account support and context, for example.

One of the easiest literacy tasks (categorised as Level 1) directs the reader to look at a medicine label to determine the “maximum number of days you should take this medicine”.

Level 5 Sample numeracy item:

Respondents were asked if it is possible to double $1000 invested at this rate after seven years and had to support their answer with their calculations. A range of responses was accepted as correct as long as a reasonable justificationwas provided, with relevant computations.

One of the easiest numeracy tasks (Level 1 required adults to look at a photograph containing two cartons of coca cola bottles and asked to find the total number of bottles in the two full cases being shown.

This Level 5 literacy item asks the reader to identify, from a complicated document taken from a page in a consumer magazine, the average advertised price for the basic clock radio receiving the highest overall score.

• ALLS does only provide a statistical ‘snapshot’ of the performance and abilities of the adult population in relation to a reading based test of literacy, numeracy and problem solving.

• Survey assessment items can only imitate real life literacy and numeracy tasks. Authenticity and validity are limited by the requirement for written responses with no allowance for oral interaction.

• There is no assessment of writing skills per se and no writing scale has been developed.

• The emphasis is on information processing via reading.

• As such, ALLS is a survey about 'aspects' of literacy and numeracy, not the whole spectrum of literacies that are part of today’s society.

• As well, the survey only deals with English and an international view of English at that - some items may not be consistent with the kinds of ways that these things are understood in Australia.

Limitations

• Provides a statistical ‘snapshot’ of the performance and abilities of the adult population.

• Reinforces the complexity of literacy/literacies/numeracies

• Comparability - can compare performances (internationally/statewise/other population categories) and look for factors and influences etc.

• Tells us something we would otherwise not know – dispel the myth of 100% literacy.

• Ammunition for the adult LLN field.

• Source of data for further research.

• Use the research and theories behind the scales – complexity of text and task, complexity of numeracy tasks, ask questions!

Uses

Number and proportion of persons in each group with skill levels 1 or 2

Prose literacy scale Document literacy scale Numeracy scale 1,000s % 1,000s % 1,000s % Australia:

7,002.9 46.4 7,066.9 46.8 7,935.6 52.5

Number and proportion of persons in each group with skill levels 1 or 2

Health literacy scale Problem solving Scale?

1,000s %

Australia:

8,980.3 59.5

ALLS – The Results

Adults with skill levels 1 or 2 by State/Territory

0

10

20

30

40

50

60

ACT NS NT Qld SA Tas Vic WA

S tate/Territory Percentage at Levels 1

and 2

ALLS – The Results

10

Our nearest neighbours: Document literacy in Oz vs NZ

0%

20%

40%

60%

80%

100%

1996 - Aust 2006 - Aust 1996 - NZ 2006 - NZ

Level 1 Level 2 Level 3 Level 4/5

-2%

+1%

No

change

+2%

- 7%

-1%

+5%

+2%

ALLS – The Results

Education vs ALLS performance

0%

20%

40%

60%

80%

100%

10 or less 11 to 15 16 to 20 21 or more

Years of formal schooling

Percentage at ALLS levels

Levels 4/5

Level 3

Level 2

Level 1

Document literacy and schooling

Every year of extra education impacts positively on performance

ALLS – The Results

11

NESB - Document literacy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1996 2006

Year

Percentage

Levels 4/5

Level 3

Level 2

Level 1

Recent migrants

Compared to 1996, of the people who migrated to Australia in the five years prior to the

survey whose first language was not English, there was a statistically significant increase

in the proportion of people attaining literacy scores of Level 3 or above on both the

prose and document scales. On the prose scale, the proportion of this group with scores

at Level 3 or above increased from 22% to 38% while on the document scale the

proportion increased from 32% to 50%

ALLS – The Results

ALLS – The Results12

Males significantly outperformed females on numeracy:

47.5% of males are at levels 1 or 2

57.6% of females are at levels 1 or 2

A difference of over 10%!

Gender - Numeracy

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Males Females

Gender

Percentage

Levels 4/5

Level 3

Level 2

Level 1

ALLS – The Results

Employment

• Employed persons had higher literacy levels on average than those who

were unemployed or not in the labour force

• Regardless of full-time or part-time status, a greater proportion of employed

persons had a skill level of 3 or higher across all scales, than either

unemployed people or those who were not in the labour force.

Participation in education and learning

• Those with lower literacy levels were less likely to have participated in

course-based learning over the last 12 months

Income

• There is a strong association between prose skill level and median personal

gross weekly income. For example, those with a skill level of 1 had a

median income of $205 less per week than those with a skill level of 2.

This gap in income potential remained fairly steady as people moved up the

skill levels. For example, the difference between those with a skill level of 2

and 3 was $192.

ALLS – The Results13

Health literacy

• In Australia, of adults whose parents‘ or guardians' highest educational

attainment was a Bachelor degree or above, 68% achieved a health literacy

Level 3 or above. This is compared to 58% of adults whose parents or

guardians completed an Advanced diploma/diploma or below.

• Half (50%) of those who reported that they had a lot of energy a good bit of

the time in the 4 weeks prior to the survey achieved a health literacy Level 3

or above.

• Of those people who volunteered in coaching, teaching or counselling, 56%

achieved a health literacy Level 3 or above.

• In contrast, of people who did not participate in any type of group or

organisation, 30% achieved a health literacy Level 3 or above.

ALLS – The Results

Post-compulsory, Vocational Education and Training & the Workforce

􀂃 There is significant evidence that the core skills of LLN are low in many of the

target groups for training and upskilling – esp. trade areas and apprenticeships

􀂃 Does this link in, for example, with the large drop out rate for apprenticeships

(~50%)?

􀂃 How do we train and teach in VET? Text-dense manuals? Online? Reading

dependent

􀂃 The age data analysis indicates that schools do not prepare students for the real

world of work

􀂃 Similarly it shows that for the ageing workforce there are also significant issues

with LLN skills and therefore the ability to be retrained

􀂃 What does it mean for the current workforce and changing practices and

expectations and access to retraining?

ALLS – The messages

Answers? There’s no single or simple solution, but there is a

known problem that isn’t going away.

14

􀂃 What are the literacy and numeracy skill requirements of training, on-the job

requirements, the content of VET courses and Training Packages and training

materials? Do we know? How do these compare with what ALLS is saying

potential learners and participants have?

􀂃 What are the implications for the training system? Which groups of adults are we

targeting in our skills shortages? What skills do they have? How do we support

them? Do we support them?

􀂃 Are VET teachers and trainers able to cope with learners with low levels of LLN?

Do we support them? How are they trained?

􀂃 And what about numeracy?

ALLS – Some key messages

Post-compulsory, Vocational Education and Training & the

Workforce: Some potential research questions

􀂃 Need more than words and the rhetoric of lifelong learning - where are our

Policies and Programs – what can we learn from overseas eg from NZ?

􀂃 It needs a joint effort: work with and form partnerships with industry and

business, unions, education and training (supported by government) – it’s not just

about the ALBE field

􀂃 Support and guidelines about LLN for the training system is needed – advice,

PD, curriculum, resources, research

􀂃 PD Support and training for the LLN field – we know the field is casualised and

marginalised and under-supported (the glory days of the 1990s when Oz was a

world leader have well and truly gone)

􀂃 PD Support and training about LLN for VET trainers. A simple solution available

NOW: Make the LLN Unit of TAA compulsory

ALLS – Some key messages

Post-compulsory, Vocational Education and Training & the

Workforce: Some possible solutions/answers? Dave’s wish list!

15

􀂃 Where are industry, business and the unions in LLN? We need to find some

champions – not just from within the ALBE sector

􀂃 Work with the school sector to address the issue of not preparing students well

for the world of work and VET –need a higher rate of completion and participation

via relevant curriculum

􀂃 And where are our Family Literacy programs?

􀂃 And what about numeracy?

ALLS – Some key messages

Post-compulsory, Vocational Education and Training & the

Workforce: Some possible solutions/answers? Dave’s wish list!

􀂃 Why is the literacy context of

health more difficult?

􀂃 And numeracy?

􀂃 What are the implications?

ALLS – Some key messages

Social Capital Outcomes – Health, Aged Care, Social Security,

etc’ Some messages and potential research questions

Panadol Elixir (1 - 5 years)

Age Average Weight Dose

1 - 2 Years 10 - 12 kg 6 - 7.5 mL

2 - 3 Years 12 - 14 kg 7.5 - 9 mL

3 - 4 Years 14 - 16 kg 9 - 10 mL

4 - 5 Years 16 - 18 kg 10 - 11 mL

5 Years 18 - 20 kg 11 - 13 mL

“The international research on ‘health

literacy’ is considerable. Studies have found

links between lower literacy and a higher

risk of hospitalisation, higher rates of

depression and an inability to understand

and comply with the use of prescription

drugs.” [Hartley & Horne, 2006, p. 7]

16

􀂃 The low LLN skills for the over 45s – this has implications for the ageing

population – staying at work longer and being expected to be more independent

– implications for health, medication and wellbeing

􀂃 Why is LLN achievement such a strong indicator of economic and social

success? In the US, people at levels 1 or 2 in numeracy are 3 times more likely

to be on social security benefits than those at the higher levels

􀂃 Research the background information from ALLS and the connections between

the range of socio-demographic and education factors/practices and the literacy

and numeracy performance of adults as measured by ALLS

ALLS – Some key messages

Social Capital Outcomes – Health, Aged Care, Financial and

Social Security, Families, etc. Some messages and potential

research questions

􀂃 And what about intergenerational literacy (and numeracy)? Australia is not active

in Family Literacy programs. Why not? Look at and support the Smith Family

model.

􀂃 Crucial for indigenous communities

􀂃 It needs a joint effort: it’s a whole of government approach and working with a

range of agencies

􀂃 What knowledge is out there about the LLN issue and the consequences?

􀂃 And what happened to Plain English and getting messages and information

across in different ways

􀂃 And what about numeracy? Financial security? Gambling? Scams? Debts?

ALLS – Some key messages

Social Capital Outcomes – Health, Aged Care, Financial and

Social Security, Families, etc. Some messages and potential research questions17

From IALS we know:

􀂃 The proportion of individuals with Level 1 skills

exerts a strong negative drag on growth in GDP

per capita so one could realise quite large

economic gains by investing in the bottom.

􀂃 The skill levels of women seem to matter more to

the growth in GDP than those of men.

Ref: Coulombe, Tramblay & Marchand (2004)

ALLS – And more …

The initial ALLS data supports other research data from the UK that

indicates the strong role that numeracy plays in both human and social

capital terms.

􀂄 People without numeracy skills suffered worse disadvantage in

employment than those with poor literacy skills alone. ... Women with

numeracy difficulties appeared especially vulnerable to exclusion from

the clerical and sales jobs to which they aspired (Bynner & Parsons,

1997, p. 27).

􀂄 For women, while the impact of low literacy and low numeracy is

substantial, low numeracy has the greatest negative effect, even when it

is combined with competent literacy. … Poor numeracy skills make it

difficult to function effectively in all areas of modern life, particularly for

women. (Bynner & Parsons, 2005, p. 7)

ALLS – The Results

ALLS – so what about numeracy?

Have we learnt anything?

For example: “it is clear from the results that when people have poor literacy

skills, they have even worse numeracy skills. The need to upgrade numeracy

skills in the context of literacy must be taken into account of in all decisions to

raise the level of adult literacy in Australia” (Wickert, 1990)

What do we do now in numeracy (and maths) teaching practices that

disadvantages women/girls so much? How do we address the gender

issues and priorities?

What policy or program actions have we taken in relation to numeracy?

Literacy versus numeracy – is it an equal partnership? Is numeracy

buried and not even bolted on? At the policy, program and funding

level? At the provider level? At the classroom level?

What are the implications for school maths curriculum?

􀂃 Australian Bureau of Statistics, (2007) Adult Literacy and Life Skills Survey: Summary results, Australia

(cat. no. 4228.0), Australian Bureau of Statistics, Canberra

􀂃 Bynner, John & Parsons, Samantha (2005) Does numeracy matter more?, National Research and

Development Centre for Adult Literacy and Numeracy (NRDC), London

􀂃 Bynner, John and Parsons, Samantha (1997) Does numeracy matter? Evidence from the National Child

Development Study on the impact of poor numeracy on adult life, Basic Skills Agency, London

􀂃 Coulombe, Serge; Tramblay, Jean-Francois & Marchand, Sylvie, (2004) Literacy scores, human capital

and growth across fourteen OECD countries, Statistics Canada, Ottawa

􀂃 Desjardins, Richard; Murray, Scott; Clermont, Yvan & Werquin, Patrick (2005) Learning a living: First

results of the Adult Literacy and Lifeskills Survey, Statistics Canada, Ottawa (downloadable through the

Statistics Canada Website at: http://www.statcan.ca/bsolc/english/bsolc?catno=89-603-XWE)

􀂃 FitzSimons, G., Mlcek, S., Hull, O. & Wright, C. 2005, Learning numeracy on the job: A case study of

chemical handling and spraying, NCVER, Adelaide.

􀂃 Gleeson, Lynne, 2005, Economic returns to education and training for adults with low numeracy skills, NCVER, Adelaide.

􀂃 Hagston, Jan, (2002) Exploring the International Adult Literacy Survey data: Implications for Australian

research and policy, Language Australia, Melbourne

􀂃 Hartley, Robyn & Horne, Jackie, (2006) Social and economic benefits of improved adult literacy:

Towards a better understanding National Centre for Vocational Education Research (NCVER), Adelaide

􀂃 Marr, Beth & Hagston, Jan, (2007) Thinking beyond numbers: Learning numeracy for the future

workplace, NCVER, Adelaide.

􀂃 T. Scott Murray, Yvan Clermont and Marilyn Binkley, International Adult Literacy Survey. Measuring

Adult Literacy and Life Skills: New Frameworks for Assessment, Statistics Canada, Ottawa

Some References

Australian Bureau of Statistics, (2007) Adult Literacy and Life Skills Survey: Summary results, Australia

(cat. no. 4228.0), Australian Bureau of Statistics, Canberra

􀂃 Bynner, John & Parsons, Samantha (2005) Does numeracy matter more?, National Research and

Development Centre for Adult Literacy and Numeracy (NRDC), London

􀂃 Bynner, John and Parsons, Samantha (1997) Does numeracy matter? Evidence from the National Child

Development Study on the impact of poor numeracy on adult life, Basic Skills Agency, London

􀂃 Coulombe, Serge; Tramblay, Jean-Francois & Marchand, Sylvie, (2004) Literacy scores, human capital

and growth across fourteen OECD countries, Statistics Canada, Ottawa

􀂃 Desjardins, Richard; Murray, Scott; Clermont, Yvan & Werquin, Patrick (2005) Learning a living: First

results of the Adult Literacy and Lifeskills Survey, Statistics Canada, Ottawa (downloadable through the

Statistics Canada Website at: http://www.statcan.ca/bsolc/english/bsolc?catno=89-603-XWE)

􀂃 FitzSimons, G., Mlcek, S., Hull, O. & Wright, C. 2005, Learning numeracy on the job: A case study of

chemical handling and spraying, NCVER, Adelaide.

􀂃 Gleeson, Lynne, 2005, Economic returns to education and training for adults with low numeracy skills, NCVER, Adelaide.

􀂃 Hagston, Jan, (2002) Exploring the International Adult Literacy Survey data: Implications for Australian

research and policy, Language Australia, Melbourne

􀂃 Hartley, Robyn & Horne, Jackie, (2006) Social and economic benefits of improved adult literacy:

Towards a better understanding National Centre for Vocational Education Research (NCVER), Adelaide

􀂃 Marr, Beth & Hagston, Jan, (2007) Thinking beyond numbers: Learning numeracy for the future

workplace, NCVER, Adelaide.

􀂃 T. Scott Murray, Yvan Clermont and Marilyn Binkley, International Adult Literacy Survey.