PROC LOGISTIC to handle Randomized Complete Block Design (RCBD)

The main idea of blocking is to ensure that the random sample you draw from the population of interest does not, by random chance, have some undesirable properties. 

*Use risk scores (riskScr) and response groups (respGrp) to create blocks;
data blocks;
set s.dem0303;
if riskScr < 650 then riskGrp=1;
else if riskScr < 750 then riskGrp=2;
else riskGrp =3;
if respScr < 260 then respGrp=1;
else if respScr < 280 then respGrp=2;
else respGrp =3;
block = compress(riskGrp||respGrp);
random=ranuni(8675309);
run;

proc sort data=blocks;
by block;
run;

*randomly assigned units to treatments within each block;
proc rank data=blocks out=blocks groups=8;
by block;
var random;
ranks treatment;
run;

*Generate design matrix (factors: interestRate sticker priceGraphic);
data blocks;
set blocks;
array factors(3) interestRate sticker priceGraphic;
do i = 1 to dim(factors);
factors(i)=substr(put(treatment,binary3.),i,1);
end;
run;

*Analysis after collecting the data;
*Block | InterestRate | Sticker | PriceGraphic;
ods select globalTests type3;
title "Block | InterestRate | Sticker | PriceGraphic";
proc logistic data=s.dem0304 namelen=40;
class block interestRate sticker priceGraphic;
model response(event="1")=
block
interestRate|sticker|priceGraphic @1
interestRate|sticker|priceGraphic @2
interestRate|sticker|priceGraphic;
run;

Note:
It is possible to get the same parameter estimates in a slightly easier to read format. The above code uses the '@' notation in the MODEL statement to arrange the terms in the model by their polynomial order. A|B|C|D in a MODEL statement yields every term from the main effects up to the three-factor interaction. 

Sample Size Calculation using PROC POWER for proportions under CRD

Q: How to ensure you get the correct sample size?

A: Leverage power analysis to determine sample size.

One of the pivotal aspects of planning an experiment is the calculation of the sample size. It is naturally neither practical nor feasible to study the whole population in any study. Hence, a set of users is selected from the population, which is less in number (size) but adequately represents the population from which it is drawn so that true inferences about the population can be made from the results obtained. This set of individuals is known as the “sample" int DOE (design of experiment).

It is a basic statistical principle with which we define the sample size before we start a study so as to avoid bias in interpreting results. If we include very few users in a study, the results cannot be generalized to the population as this sample will not represent the size of the target population. Further, the study then may not be able to detect the difference between test groups, making the study unethical. On the other hand, if we study more subjects than required, we put more users than needed, also making the study wasting precious resources, including the researchers’ time.

Generally, the sample size for any study depends on the:

•  Acceptable level of significance
•  Power of the study
•  Expected effect size
•  Expected variance

Level of Significance

This is the pre-defined threshold value to reject the null hypothesis. Usually p<0.05 will be taken as statistically significant to accept that the result is observed due to chance. To put in different words, it is possible to accept the detection of a difference 5 out of 100 times when actually no difference exists (i.e., get a false positive result). It is denoted by letter α(alpha) as Type 1 error.

Power

Exactly conversely, type II of error indicate failing to detect a difference when actually there is a difference (i.e., false negative result). The false negative rate is the proportion of positive instances that were erroneously reported as negative and is referred to in statistics by the letter β(beta). The power of the study then is equal to (1 –β) and is the probability of failing to detect a difference when actually there is a difference. The power of a study increases as the chances of committing a Type II error decrease. Usually most studies accept a power of 80%. This means that we are accepting that one in five times (that is 20%) we will miss a real difference. 

Expected effect size

The effect size is the minimum deviation from the null hypothesis that the hypothesis test set up to detect. Suppose you are comparing different responses between treatment group and control group, and the measuring metrics are group means denoted as μ1 and  μ2,  the expected effect size equals to abs( μ1- μ2). Generally speaking, the smaller the difference you want to detect, the larger the required sample size.

Expected variance 

The variance describes the variability/noise among measurement units. As variance gets larger, it gets harder to detect a significant difference, so the bigger sample size requires. The pilot study or similar study could help to determine the expected variance.

Reference

1. Design and Analysis of Experiments. by Montgomery, D.C.X (Hoboken, New Jersey: John Wiley & Sons, Inc., 2000)

* Power with One Sample; 

proc power;
oneSampleFreq
/* hypotheses of interest */
nullProportion=0.1
proportion=0.2
sides=1
/* decision rule */
alpha=0.3
/* sample size */
nTotal=10 20 25 30 40 50 100
power=.
;
run;

*normal approximation;
proc power;
oneSampleFreq
/* hypotheses of interest */
nullProportion=0.1
proportion=0.2
sides=1
/* decision rule */
alpha=0.3
/* solve for sample size */
nTotal=.
power=.80
/* different distributional assumptions */
test=z
method=normal
;
run;

* Power with Two Samples;
** SAS
proc power;
twoSampleFreq
/* hypotheses of interest */
refProportion=0.01
proportionDiff=0.001/*first factor difference*/ .0025/*second factor difference*/
sides=1 2
/* decision rule */
alpha=0.05
/* sample size */
nTotal=.
power=.6 .99
;
plot y=power
yopts=(crossref=YES ref=.8)
vary(color by proportionDiff,
symbol by sides);
run;

proc power;
twoSampleFreq
/* hypotheses of interest */
refProportion=0.1
proportiondiff=0.1
sides=1
/* decision rule */
alpha=0.05
/* sample size */
nTotal=.
power=.6 .8 .99
/* how balanced the test is */
groupWeights=(1 1) (10 15) (1 2) (1 3) (1 10)
;
plot;
run;

** R
install.packages("pwr")
require(pwr)

delta <- 20
sigma <- 60
d <- delta/sigma
pwr.t.test(d=d, sig.level=.05, power = .80, type = 'two.sample')

PROC POWER, PROC GLMPOWER for proportions under CRD

* Balance with Two Factor; 

proc power;
twosamplefreq
refproportion= 0.010
proportiondiff=0.001 0.0025
ntotal=.
power=.8;
run;

data work.a;
input intro $1-4
goto $6-9
responseRate;
variance= responseRate*(1-responseRate);
datalines;
LOW LOW .0135
LOW HIGH .0125
HIGH LOW .011
HIGH HIGH .010
;
run;

proc glmpower data=work.a;
class intro goto;
model responseRate=intro|goto;
power
power=.8
ntotal=.
stddev=%sysfunc(sqrt(.01*.99))
%sysfunc(sqrt(.011*.989));
run;

* Three Two-Level Factors;
* 8 trts;
data work.dem0302;
do a=-1 to 1 by 2;
do b=-1 to 1 by 2;
do c=-1 to 1 by 2;
respRate=.01+.00050*sum(a,b,c);
output;
end;
end;
end;
run;

proc print data=work.dem0302;
run;

proc glmpower data=work.dem0302;
class a b c;
model respRate=a|b|c;
power
power=.8
ntotal=.
stddev=%sysfunc(sqrt(.01*.99));
run;

*Analysis Examples:
data work.ex0302;
informat responseRate percent6.4;
input interestRate $1-4
sticker $6-8
priceGraphic $10-14
responseRate;
datalines;
LOW YES SMALL 1.00%
LOW YES LARGE 1.20%
LOW NO SMALL 0.85%
LOW NO LARGE 1.00%
HIGH YES SMALL 0.80%
HIGH YES LARGE 1.00%
HIGH NO SMALL 0.60%
HIGH NO LARGE 0.75%
;
run;

*N=4,973,040;
proc glmpower data=work.ex0302;
class interestRate Sticker priceGraphic;
model responseRate=interestRate|Sticker|priceGraphic;
power
power=.8
ntotal=.
stddev=%sysfunc(sqrt(.01*.99));
run;

proc logistic data=s.two3;
class interestRate Sticker priceGraphic;
model orders/volume=
interestRate|Sticker|priceGraphic @1
interestRate|Sticker|priceGraphic @2
interestRate|Sticker|priceGraphic;
output out=work.plot p=p l=l u=u;
run;